MACHINE VISION APPLICATIONS

FOR AGRICULTURE

Dr. P. SUDHAKARA RAO

DEPUTY DIRECTOR
CEERI CENTRE, CHENNAI
MACHINE VISION TECHNOLOGY

APPLICATIONS FOR AGRICULTURE

• Machine vision systems not only recognize

size, shape, color, texture of objects but
also provide numerical attributes of the
objects or scene being imaged
• Land-based and aerial-based remote
sensing help natural resources
assessments, precision farming, post
harvest product quality and safety
detection
• Classification and sorting

4/17/2018 2
MACHINE VISION TECHNOLOGY

IMAGING LIGHT REGIONS

• Most common Machine vision systems image objects in the
visible color region, while some systems image them in
ultraviolet (UV), near-infrared (NIR) and infrared (IR)
regions.
– to determine pre-harvest plant maturity, disease, or stress
states.
– to detect post harvest quality and safety such as defects,
composition, functional properties, diseases and
contamination of plants, grains and nuts, vegetables and fruits
and animal products.
– to determine plant and vegetable variety, maturity, ripeness,
and quality.
• X-ray imaging and nuclear magnetic resonant imaging
(MRI) are some times used to detect internals of agriculture
produce.

4/17/2018 3
MACHINE VISION TECHNOLOGY

SOME ADVANTAGES

• The technology is fairly accurate,

nondestructive and yields consistent
results.
• Machine vision discussed here is limited
to camera machine vision systems.
• It holds great potential and benefits for the
agricultural industry
– Simplicity, Low cost, Rapid inspection rate,
and broad range of applications.

4/17/2018 4
MACHINE VISION TECHNOLOGY
COMPONENTS OF A MACHINE VISION
SYSTEM

• A basic machine vision system consists of

a camera, a computer equipped with an
image acquisition board and illuminating
system
• In agricultural applications acquire
reflectance, transmittance or fluorescence
images under UV, VIS, or NIR illumination.
• Computer software acquire images and
performing processing and storage of the
images.
4/17/2018 5
MACHINE VISION TECHNOLOGY

FRAME GRABBER

• Monochrome frame-grabber boards are capable of taking

RS-170 or CCIR video inputs
• The color frame-grabber receives NTSC, PAL, or S-VHS
input signals.
• Advanced grabbers adopt different standards like Camera
link and IEEE-1394 (Fire Wire)
• The card will perform image acquisition (digital or analog) ,
camera control, and image data pre-processing
• Advanced grabber boards preprocess images
– ‘‘first-in-first-out’’ (FIFO)
– ‘‘look-up table’’ (LUT).
– communicate with the host CPU’s memory
– On-board DSP/FPGA

4/17/2018 6
MACHINE VISION TECHNOLOGY

LIGHTING
• The incident light shall present the objects or scenes
in the optimal way to be recognized or analyzed.
• The lighting unit selection and its configuration in a
machine vision system depend on the application,
UV (200_/400 nm), VIS (400_/700 nm), or NIR
(700_/2500 nm), thermal imaging ( > 2500 nm).
• The light may be transmitted, reflected, or absorbed
by object.
• The absorbed light can also be re-emitted
(fluorescence).
• A number of compounds emit fluorescence in the
Visual region when excited with UV radiation. This is
an integrated function of the angle and wavelength
of the incident light and chemical and physical
composition of the object.
4/17/2018 7
MACHINE VISION TECHNOLOGY
IMAGE PROCESSING AND
ANALYSIS SOFTWARE

• Manipulates information within image to make it useful.

• Image processing in agricultural applications may consist
of three steps:
– image enhancement: morphological operations, filters,
and pixel-to-pixel operations
– Image feature extraction: Statistical procedures like
mean, standard deviation, and variance to more
complex measurement such as principle component
analysis are used to extract features from digital
images.
– Image feature classification: Numerical techniques
such as neural networks and fuzzy inference systems
can be successfully applied to perform image feature
classification.
4/17/2018 8
MACHINE VISION TECHNOLOGY

CONTOUR IMAGE GENERATION

[CEERI SOFTWARE]

Image With Image With Back

Back Ground Image Contour
Ground Removed
4/17/2018 9
MACHINE VISION TECHNOLOGY

CCD CAMERAS
• Quality images are required to identify features.
• An imaging camera converts the light from the object
into electrical signals using a charge-coupled device
(CCD).
• Monochrome imaging is adopted to perform simple
operations like shape and size recognition.
• Multi spectral cameras are used for detection of
surface defects and diseases on meat, grains, fruits,
and vegetables, etc.
• Efficient hardware and software are required to
process the images fast enough for on-line
implementation.

4/17/2018 10
MACHINE VISION TECHNOLOGY

MONOCHROME IMAGING
• Monochrome imaging requires a single-
chip CCD and can sense Visual and NIR.
• The resolution (number of pixels) of a CCD
depends on the nature of applications.
• Monochrome imaging along with colour
imaging is used for the detection of
blemishes, bruises, scars, cracks in fruits
and vegetables.

4/17/2018 11
MACHINE VISION TECHNOLOGY

COLOR IMAGING

• A single chip CCD is also used for color

imaging by by alternating the pixels for red,
green, and blue (RGB).
• Three chip CCD uses one chip for each
primary colour, which is accomplished by
using a prism assembly with band pass
filters.
• Color difference between bruised and non-
bruised regions can be used for bruise
detection, classify livers and hearts of
wholesome and unwholesome chickens.
4/17/2018 12
MACHINE VISION TECHNOLOGY
3-CHIP CCD CAMERA

• Each CCD in a three chip camera receives RGB

colors to produce near the true color images of the
object.
• The image acquired by each CCD is a
monochromatic either for red, green or blue and
hence composition of the three-channel signals
provides a near true color image of the object.
4/17/2018 13
MACHINE VISION TECHNOLOGY

HYPERSPECTRAL IMAGING
• Combines imaging and spectroscopy.
– Each pixel on the image surface possesses a spectral
signature of the object at that pixel.
• Two general approaches:
– Capture a series of narrow-band spectral images to
accomplish a three-dimensional image cube.
– Pushbroom method where a line of spatial information with
a full spectral range per spatial pixel is captured
sequentially to complete a volume of spatial-spectral data.
Typically this method uses a line scan camera with
odditional optics.

4/17/2018 14
 SOME HYPERSPECTRAL
MACHINE VISION TECHNOLOGY

APPLICATIONS
Identify materials and detect subtle and/or
minor features in an object.
Precision agriculture- detection of plant
stress or crop infestation.
Agricultural product quality and safety
sensing.
Applied to measure soluble solids
distribution in fruits.
Capable of both airborne and
ground/laboratory data acquisitions using a
LCTF (450 to 750 nm).
4/17/2018 15
MACHINE VISION TECHNOLOGY

MULTISPECTRAL IMAGING
• Multispectral imaging consists of a set of
several images, each acquired at a narrow
band of wavelengths using
– Liquid crystal Tunable filter (LCTF)
– Acousto-optic Tunable filter (AOTF)
– Multiple filters.

4/17/2018 16
MACHINE VISION TECHNOLOGY
SOME APPLICATIONS OF
MULTISPECTRAL IMAGING

• Three wavelengths, 670, 800, and 990 nm,

are used to detect healthy and unhealthy
leaf tissues.
• Spatial information at eight wavelengths
are used to detect 15 kinds of blemishes
on a potato.
• Used for detecting unwholesome poultry
carcasses.
• Used to characterize chicken heart images
for disease detection.
4/17/2018 17
MACHINE VISION TECHNOLOGY
MULTIPLE FILTER METHOD FOR
MULTISPECTRAL IMAGING

4/17/2018 18
MACHINE VISION TECHNOLOGY

MULTISPECTRAL IMAGING

• Simultaneous RGB color and

near infrared imaging for
color sorting and bruise
detection on apples.
• A 2D multi spectral images
of the object on a single
detector array without any
moving parts, such as
scanning gratings or rotating
filter wheels.
• Depending on the application
this image configuration can
be modified to acquire 2 to 8
separate, spectral images.

4/17/2018 19
MACHINE VISION TECHNOLOGY

Unique Multi spectral imaging technique

[CEERI System]

4/17/2018 20
MACHINE VISION TECHNOLOGY
APPLE SORTING MACHINE
CEERI DEVELOPMENT

4/17/2018 21
MACHINE VISION TECHNOLOGY

Product Washing & Waxing

Product Singulation
Product Rotation system & Conveying
Sorting, Grading, Packing and Labeling
Uniform Illumination
Multiple Products and Multiple Images
(Wavelengths)
Optical system for BLOB Analysis
5 Products per second

4/17/2018 22
MACHINE VISION TECHNOLOGY

Suitable wide Angle lens to cover and

capture More then one products (Three to
Six) at a time
Special prism arrangement for obtaining
three images at three specific wavelengths
Colour CCD camera monitors the scene
Product synchronization interface will
command the computer to capture the
image
The image is transferred to the embedded
system for analysis

4/17/2018 23
MACHINE VISION TECHNOLOGY

Large number of manually Graded Products

are chosen and numbered
Many portions (AOI) of each product’s
surface are collected by the system –
Unlimited number of AOIs
The training process ends after exhausting
the graded products
The training process to be verified by placing
the product/s of known grade and cross
check

4/17/2018 24
MACHINE VISION TECHNOLOGY
ANALYSIS IN RGB / HSI COLOUR SPACE

4/17/2018 25
MACHINE VISION TECHNOLOGY
TRAINING THE SYSTEM

3 APPLES IN ONE
CAPTURE
SIX POSTURES
FOR EACH
APPLE

4/17/2018 26
MACHINE VISION TECHNOLOGY

GRADING PROCESS

4/17/2018 27
 28
4/17/2018
MACHINE VISION TECHNOLOGY
 29
4/17/2018
MACHINE VISION TECHNOLOGY
 30
4/17/2018
MACHINE VISION TECHNOLOGY
 31
4/17/2018
MACHINE VISION TECHNOLOGY
 32
4/17/2018
MACHINE VISION TECHNOLOGY
MACHINE VISION
TECHNOLOGY

TYPICAL APPLICATIONS

4/17/2018 33
MACHINE VISION TECHNOLOGY

TOMATO INSPECTION SYSTEM

4/17/2018 34
MACHINE VISION TECHNOLOGY

TOMATO RIPENESS

Multi-spectral images of 4 different variety of

tomatoes (12 stages ). PCA and LDA are used for
training and classification
4/17/2018 35
MACHINE VISION TECHNOLOGY
NIR Imaging for Apple Skin Defects
Detection
Second Difference Image Mask created Uncontaminated
chlorophyII Image 685, 720 and With NIR band Apples showed no defects
absorption 870 nm.

band in the red white spots White spots are tiny

at 685 nm are defects and bruised spots
contamination
4/17/2018 36
MACHINE VISION TECHNOLOGY
RATIO IMAGES

Ratio between two reflectance images at 750
and 800 nm bands followed by simple
threshold could differentiate thick patches of
manure from region of uncontaminated
apples.
Simple ratio of two fluorescence bands
at 680 and 450 nm could easily
differentiate uncontaminated portions
of apple surface from contaminated
spots, regardless of apple skin color
and thickness of manure treatments.

4/17/2018 37
MACHINE VISION TECHNOLOGY

Defect identification in dried pears

• Images of the
dried pears.
• Suitable
thresholding
technique is
used to identify
the defects in
the pears.

4/17/2018 38
MACHINE VISION TECHNOLOGY

POTATO INSPECTION SYSTEM

• Two conveyor belts placed one after

the another, transport the potato
under the camera for inspection.
• Mirrors are placed between the
conveyors to obtain a 360 degree view
of the potato.
• Four lamps illuminate the bottom part
of the potato and six lamps illuminate
the upper part of the potato.
• A digital 3 CCD color line camera is
used for imaging which has got
resolution of 2 pixels/mm.
• The system identifies the color, size ,
shape and defects of the potatoes.
• Four different defects classes of the Four different defects
potatoes.
4/17/2018 39
MACHINE VISION TECHNOLOGY
BELL PEPPER DEFECTS
IDENTIFICATION
• Defect detection requires
identification of a specific
feature on the product.
• Quality grading often requires
classification of a specific
attribute such as color which
may relate directly to ripeness.
• Automation of this process
requires high quality color
imaging that can accurately
detect subtle variations in
shade.
• Both visible color and NIR
spectral imaging are required.

4/17/2018 40
MACHINE VISION TECHNOLOGY

CITRUS FRUIT GRADING

AND DEFECT IDENTIFICATION

• Various defect
images of the citrus
fruit.
• The grading system
uses an array of low-
cost color cameras
to identify blemishes,
defects, and color
inconsistencies in the
fruits.

4/17/2018 41
MACHINE VISION TECHNOLOGY

Machine vision system for sorting and

grading purpose

4/17/2018 42
MACHINE VISION TECHNOLOGY

LENTIL SEGREGATION (Training)

• Visual appearance of various

lentil types indicates that seed
size and color are good
features for segregating
different lentil types.

• The mean color(hue) measured

from the image represent the
overall color of the bulk
samples.

4/17/2018 43
MACHINE VISION TECHNOLOGY

LENTIL SEGREGATION

• Mean seed size (area)

and mean color of the
seeds in the image
provide sufficient
information to segregate
all five types of lentils.

• A rule-based classifier
type identifies the type
based on mean seed
area and colour.

4/17/2018 44
MACHINE VISION TECHNOLOGY
HYPERSPECTRAL IMAGING SYSTEM
FOR FOOD SAFETY STUDY

4/17/2018 45
MACHINE VISION TECHNOLOGY
Typical hyperspectral image data for the
contaminated and uncontaminated surfaces
of the apples.
In general uncontaminated apple surface showed higher
reflectance in the visual region ( > 600 nm) and NIR regions
compared to the defective or contaminated surfaces except for
the bruise spots, which had higher reflectance.
Area with scabs exhibited the lowest reflectances.

4/17/2018 46
MACHINE VISION TECHNOLOGY

Intelligent machine vision systems

• Selective harvesting, machine guidance, plant identification, and plant

health assessment.
assessment.
• Spectral reflectivity varies widely in the near infrared spectrum and is
closely tied to the presence of chlorophyll and xanthophylls which are
related to plant health.
health.
• Color infrared (CIR) imagery is based on green, red, and near infrared
wavebands that are mapped to the blue, green, and red components of a
typical color display.
display.
• Due to the wide variability of the infrared component, the resulting image
amplifies small differences in color making it easier to detect low levels
of disease in crops.
crops. CIR imagery of natural vegetation provides the same
advantages, enabling assessment of environmental damage such as acid
rain..
rain
• Intelligent machine vision systems that can identify specific areas of
infestation and therefore direct application of sprays can have a
tremendous effect on the economies of agriculture.
agriculture. Spectral weed
detection is useful for both large scale field mapping and real time
tractor based spray control
control..

4/17/2018 47
MACHINE VISION TECHNOLOGY

Intelligent machine vision systems

4/17/2018 48
MACHINE VISION TECHNOLOGY

HYPER SPECTRAL IMAGING

Find applications in earth
remote and medical diagnosis.
The techniques combine the
features of imaging and
spectroscopy to acquire both
spatial and spectral imaging
information from an object.
This technique yields much
more useful information than
other imaging techniques
because each pixel on the
image surface possesses a
spectral signature of the
object at that pixel.

A line of spatial information with a full spectral range per spatial pixel is
captured sequentially to complete volume of spatial-spectral data
The advantages of this type of system is that sample sizes in one of the
spatial directions are not limited by the sizes of CCD.
4/17/2018 49
MACHINE VISION TECHNOLOGY
ON-LINE POULTRY INSPECTION BY
A MULTI-CAMERA SYSTEM
These systems are able to detect and
identify carcasses unfit for human
consumption.
A total of 15 horizontal layers are
generated from each segmented
image.
For each layer a centroid is Front - 540 nm Front - 200 nm
calculated from the binary image.
Based on these centroids, each layer
was divided into several square
blocks [ 16 x 16 pixels ] for a total of
107 blocks.
The intensities variations in each of
the block are studied with respect to Back - 540 nm Back - 200 nm
wavelength region of 540 and 700 nm.

4/17/2018 50