LabVIEW Machine Vision for Line Detection

EGR315: Instrumentation
Spring 2010
Than Aung and Mike Conlow
Department of Physics and Engineering
Elizabethtown College
Elizabethtown, Pennsylvania
Email: aungt@etown.edu, and conlowm@etown.edu.

Abstract – To improve the previous

package a prototype virtual instrument was
development of a visual obstacle
developed to attempt to improve processing
avoidance algorithm without using
speed by using LabVIEW for the entire
proprietary software that is not
image processing procedure through a USB
financially practical to purchase. The
webcam [6].
goal is to use more advanced methods to
There were several improvements that
produce more concise output in terms of
needed to be made to the prototype in order
turning angle and the nearest point of
to justify its implementation over the
interest.
previous vision system. The turning
I. Introduction
algorithm depended on a set of line
The following data is an analysis of
detection sub virtual instruments that
improvements made to a system previously
generated large amounts of noise due to
developed using NI Vision Development
inadequate intensity filtering. To resolve
software to detect white lines on non
these and other issues the filtering,
uniform grass. The need for this arose due to
thresholding, and line detection were
over complexity of a vision system that is on
programmed using base package of
an autonomous robot that is used as a
LabVIEW, using LabVIEW IMAQ and
learning platform. The current system [5]
IMAQ USB to capture the images from a
uses a DVT Legend 554C that collects and
webcam [6].
filters the images internally and transmits
The result is a great improvement over the
the relevant data, via TCP/IP string, to a
previous version. Further enhancements still
LabVIEW program that is a closed loop
need to be implemented in order to operate
motor control. During the fall semester of
properly in the field, but the goals that were
2009 using the NI Vision Development
set for this semester have been met.
 II. Background
The previous project mainly employed NI
Vision Development Module 9.0 (Trial
Version), which provides various image
processing and machine vision tools. By
using the edge-detection sub
instrument we implemented the following
line detection algorithm.
The image resolution is set to 320x240
pixels capturing at 8 frames per second.
Each frame is then converted to an 8 bit
gray-scale image and then the image is
segmented into regions as follows [6]:
Figure 1: Edge Detection Regions
White lines are detected with IMAQ Edge
Detection by finding lines in the eight
boarder regions represented in green in
Figure 1. In our algorithm, we use two
vertical lines (VL1 and VL2) and two
horizontal lines (HL1 and HL2) detectors.
VCL (Vertical Center Line) is
calculated by averaging VL1 and VL2.
Likewise, HCL (Horizontal Center Line) is
calculated by averaging HL1 and HL2. The
line angle is then calculated by find the
angle between HCL and VCL, by using
Where m2 is the slope of HCL and m1 is
virtual
the slope of VCL. By using the intersection
point and the angle between HCL and VCL,
the appropriate heading for the robot is
determined.
Although the algorithm seems simple
enough, there are a lot of drawbacks. First,
when converting from 32 bit color images to
8 bit gray-scale images there is a loss of
edge information in every frame. In the
presence of background noises it is very
difficult to detect stable edges, thereby
making line detection less accurate. Second,
using four edge detectors is unnecessarily
redundant, and over-use of edge detectors
results in slower processing. Third, we did
not have time to implement the filters to
eliminate the noises and to threshold the
unnecessary pixels information. Finally,
since we used the 30-day trial version of NI
Vision Development Module, to continue
using the program the only option was to
purchase the three thousand dollar full
version.
then
Therefore, the primary motivation of our
project was to solve the problems we faced
by using NI Vision Development, and amount of blue the most intense blues would
improve upon the shortcomings of the first be whites.
project. With the main goals in mind, we In binary format, 32 bits color is
developed the second version of our line represented as follows:
detection algorithm. Alpha Red Green Blue
III. Implementation xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx

Our project goals were to reduce the

noises during the image acquisition, enhance (x is a binary 1 or 0). In order to extract Blue
the edge information, and stabilize the color information, we performed an AND
detected line even with the background operation with the 32 color bits with the
reflections and light sources. Therefore, we following binary bit mask [2].
divided the project into different modular 0000 0000 0000 0000 0000 0000 1111 1111

processes to achieve these goals.

A. Single Color Extraction
The images acquired from the camera
(Creative VF-0050) are 320x240 32-bit
color images. Although we can simply
convert the 32-bit color (RGB) images to 8-
bit grayscale images by averaging the 32-bit
color, we have learned how to use a better
Figure 2: 32-bit Color Image
method for eliminating the noises and
enhancing the edge information. Since the
background of the images is mostly green,
we decided that if we just simply extract the
blue color pixels from RGB the images, we
can reduce the noises and enhance the white
color lines. The thought process behind this
is that the dirt and grass are mostly
Figure 3: 8-bit Blue Color Image
composed of reds and greens, so if we were
to only look at objects composed of some

Figure 4: Blue Color Extraction
By extracting the blue color plane from an
image only pixels with a high blue intensity
will appear white. This will reduce some
noise from high intensity greens and reds.
To eliminate noise from natural reflections a
spatial convolution using a flattening filter
will be used to further enhance the image
edges.
B. Spatial Convolution Filter
To prevent large quantities of noise it was
necessary to implement a convolution using
a 7x7 flatten kernel [2]. Since the image is
represented as a two dimensional matrix of
blue intensity values, after the color
extractions, a convolution using a kernel of
all ones will be applied to the image
reducing high level intensity values. The
reason this is done after the blue plane is
extracted is to prevent the high intensity
greens and reds from mixing with blues
causing the blue plane to be inaccurate when
extracted.
Figure 5: LabVIEW Convolution
The figure above is the virtual instrument
for a convolution, where X is the image
matrix and Y is the convolution kernel.
Since there are many convolutions being
performed the algorithm is using frequency
domain convolution. The way the operation
is performed first requires that the image
matrix is padded horizontally and vertically
by one minus the width and height of the
kernel [4]. Then by shifting the kernel over
the image the padded matrix is given the
values from the convolution, which is the
Fourier Transform of X and Y multiplied
together as a dot product in terms of two
dimensional matrices and then the result is
transformed back to give the value desired
in the original resolution of 320x240 [4].
The following figure is a representation of
how special convolution is utilized to apply
a kernel to a simple set of data.
Figure 6: Padding and Filtering
 From the resulting matrix it is possible to of each intensity value. Once we know the
get the average value of every 7x7 value by intensity values of the image and their
dividing the elements by forty-nine to get frequencies, it will become easier for us to
the values in the matrix in terms one byte determine the edges that we are interested.
per index. Now all the high intensity noise
that would have thrown off the later line
detection should be gone as long as the noise
doesn’t appear in large groups.
From the flattened image it should now be
much easier to find the white lines.
However, since the image was flattened the
line edges will not be as intense as they
were. So the next step is to determine the Figure 7: Intensity Histogram
highest intensity values in the image to In Figure 7, we can see clearly that the
attempt to only detect the highest intensities maximum image intensity is around 200,
in a set range. and minimum image intensity is around 15.
C. Intensity Analysis However, even with the different
After the single-color extraction and background lighting, there is one thing we
filtering, the image seems to be ready for know for sure: the white lines always have
edge detection. However, there is still one the maximum intensity. Therefore, if we can
problem we have to solve before performing extract the intensity range from 180 to 200,
the edge detection. Under non-uniform we can detect the white lines of the image.
background lighting, the maximum image
intensity and intensity distribution of the
image change accordingly, and it is almost
impossible to perform normal thresholding
to detect the edges. Therefore, we need to
analyze the image intensity distribution. In
order to do so, we first tried to acquire the
intensity histogram of the image, which
includes both intensity range and frequency
 this case we will use a variable threshold
value, which will be adjusted according to
the background lighting as discussed in the
previous section. Since we already know the
maximum intensity of the image from the
intensity analysis, then we will calculate the
variable threshold as follows:

Figure 8: Intensity Analysis

In Figure 8 it can be seen that the highest
value found in the histogram is being passes
on to the next part of the program. Also at
the bottom of the figure there is a user Figure 9: Adaptive Thresholding

control called Interval that contains the

range of intensities accepted as white. This
is where the adaptive threshold will receive
its maximum and intensity range from.
D. Adaptive Thresholding
Actually, thresholding is the simplest most
powerful method for image segmentation.
Mathematically, thresholding can be Figure 10: Thresholded Image/ Interval = 20

described as [1]: E. Hough Transformation

, where f(x,y) is the input image, g(x,y) is
the threshold output image and T the
threshold. Generally, thresholding uses the
fixed value of T to segment the images. In
Once we get the edge pixels after the
adaptive thresholding we need to link them
together to form a meaningful line. To
accomplish this task a Hough
Transformation will be used to bridge any
gaps in the line that may appear. This will
give us the position and direction of the line
in the field of view [1] [2].
In Hough Transformations, each pixel (x,
y) is transformed from Cartesian Space to
Hough Space, H (R,θ), as follows:

, where 0 < R < and

. If two pixels (x1, y1) and (x2, y2) are Figure 11: Hough Space (R, θ)
co-linear, we will get the same value for R Once we get the R and θ, we need to shift
and θ. In other words, a line in Cartesian them as follows:
Space is represented as a point in Hough
Space. A simple Hough Transformation can
be achieved by using a two-dimensional
accumulator (array), which corresponds to R
and θ. Each cell of the accumulator is
defined by unique R and θ values, and the
value inside each cell is increased according
to the number of co-linear points in
Cartesian space. However, for practical
purposes, this algorithm is too slow for real-
time image processing. Therefore, we must
use the Matlab ‘hough function’ for our line-
detection [2].
Once the accumulator is filled, we look for
the maximum cell value stored in the
accumulator and its related R and θ. The
resulting R and θ represent the line we are
interested. The most efficient way to implement these
equations is to use a formula node.
 Once the line is generated in the Hough
Transformation the line values are sent to a
line detection algorithm to determine how to
properly handle the possibility of crossing
the line.
F. Line Detection Algorithm
Once we get the values of x, y, x1 and y1,
we use the following
algorithm to calculate the line angle. This
will be used to determine whether the robot
needs to turn, along with what direction and
intensity the turn should be made.
IV. Results and Performance Analysis
In order to test the reliability and
performance of our algorithm, we carried
out a series of tests with different scenarios.
The results were captured and are shown
below; for each set of conditions there is a
picture of what the camera sees followed by
line detection what the program interprets as the proper
avoidance maneuver along with where the
nearest line is.

If (x > 0) AND
Yes
(y > 0)

No

If ( α < 30) AND

(α > 0)
If ( α < 30) AND
(α > 0)
No
No

Yes
Yes

Go Straight Turn Left Go Straight Turn Right

Right = x1 > 160

Left = Not (Right)

Figure 12: Turning Algorithm

Right will tell us if the detected line is
Test 1: Simple Right
located on the left side of the camera and
Left will tell us if the detected line is located
on the right side of the camera. These are
decided by what x coordinate pixel the
nearest line is on at the bottom row of the
image.
Test 2: Right w/ Obstacle
Test 4: Simple Left

Test 3: Parallel
 Test 5: Left w/ Obstacle
According to the test results, we found that
our new algorithm can give more accurate
and reliable results than our old algorithm.
In addition, since we do not use NI Vision
Development, and wrote the whole project
with the intrinsic LabVIEW functions, we
also solved the problems related to software
expiration. One problem that still needs to
be dealt with is if no line is present the
adaptive threshold will give the line
detection of the largest set of intensities.
This has to be solved before the system can
the system would need the guarantee that
either the left or right line would be in the
field of view. For our purposes this is not an
acceptable loss, but the improvements made
are still enough to show definite
improvement over the previous prototype.
VI. References
1. Davies, E.R. Machine Vision. 2nd ed. San Diego Academic
Press., 1997. 80-269.
2. González, Rafael C.; Woods Richard Eugene; and Eddins
Steven L. Digital Image Processing with MATLAB. Pearson
Prentice Hall., 2004.380-406.
3. Jahne Bernd. Digital Image Processing. 6th ed. Heidelberg:
Springer-Verlag., 2005.331-340.
4. "NI Vision Acquisition Software." National Instrument. 30 Nov
2009. http://sine.ni.com/psp/app/doc/p/id/psp-394 >
5. Painter, James G. Vision System for Wunderbot IV Autonomous
Robot. Elizabethtown College, 9 May 2008.
6. Aung, Than L. & Conlow Michael. Alternative Vision System
for Wunderbot V. Elizabethtown College, 9 Dec 2009.

be declared a fully functional obstacle

avoidance utility.
V. Further Improvements
Although our algorithm is satisfactory to
some extent, there is a lot to be improved
upon that would require more time and a
budget plan for additional equipment. First
of all, we use monocular vision system, to
detect the lines. By adding a second camera
the system could be reprogrammed to have
one camera handle the left and the other
dedicated to the right line. This would allow
greater control due to the visual field being
doubled.
For the project to be a feasible substitute
for the current system the algorithm will
need the ability to distinguish whether or not
a line is even present. Without this ability