Sunteți pe pagina 1din 10

Recent Patents on Mechanical Engineering 2009, 2, 169-178 169

1874-477X/09 $100.00+.00 2009 Bentham Science Publishers Ltd.


Recent Innovations in Off-Road Intelligent Vehicles: In-Field Automatic
Navigation
Francisco Rovira-Ms*
School of Agricultural Engineering, Polytechnic University of Valencia, Valencia, Spain
Received: March 25, 2009; Accepted: May 27, 2009; Revised: May 29, 2009
Abstract: The fast evolution of computing power together with the improvement of electronics and sensors in the last
decade has resulted in the introduction of vehicle automation at a consumer level; therefore, every day, an increasing
amount of commercial solutions are reaching the global market. The special shape in which agricultural fields are
structured, on the other hand, makes off-road vehicles one of the best candidates for driving assistance. Very often, tractor
or harvester operators are required to follow predefined rows for long periods of time, leading to physical and mental
fatigue that can provoke dangerous situations or even fatal accidents. In such conditions, the aid brought by intelligent
systems can represent a positive asset to farm production systems. The recent popularization of satellite navigation
systems, such as GPS, has meant the final thrust to the deployment of automatic guidance systems. However, not only
global positioning sensors are key in vehicle automation but a complex architecture comprising imaging, laser and
ultrasonic devices can be found in recent models as well. This article presents an overview of patents issued in this field of
automotive engineering over the last twenty years.
Keywords: Automatic guidance, off-road vehicles, GPS auto-steering, farm machinery automation, autonomous navigation,
field robotics, agricultural robotics, crop-tracking vehicles, tractor guidance & intelligent vehicles.
1. BACKGROUND AND SCOPE
Automatic guidance of vehicles is one of those themes
where imagination and popular imagery has long preceded
technology. As early as 1924, Willrodt described a system to
guide a machine following furrows across a field, and
interestingly enough, it was published as a patent [1]. Since
the time of that pioneering work, engineers have been lucu-
brating and imagining ways to make vehicles move without a
driver. However, the solutions found were quite limited and
specifically adapted to particular environments. The silicon
revolution and digital era of the sixties and seventies was
going to change the scene; the fast growth of technology in
computers and electronics soon hinted that the old dream of
autonomous robots might be feasible. Yet there was a long
way to go, but the special know-how that automatic navi-
gation requires was being developed at a high pace. Never-
theless, not until the eighties did ideas start to take form with
the first prototypes beginning the desired trip from concept
development to product development. The definite stroke has
been banged by the popularization of the Global Positioning
System (GPS), which allows a real-time localization of a
mobile platform in a global frame. With all the ingredients
for automation obtainable, it makes sense to wonder if there
is still a strong interest in the automation of off-road vehi-
cles. The answer is affirmative, and the increasing number of
patents and scientific articles focusing on these topics
demonstrates that they are of intense and immediate interest.
Spatial variability is a typical phenomenon in agriculture
which traditional farming systems find difficult to cope with.

*Address Correspondence to this author at the School of Agricultural
Engineering, Polytechnic University of Valencia, Camino de Vera, s/n,
46022 Valencia, Spain; Tel: +34-963877291; Fax: +34-963877299;
E-mail: frovira@dmta.upv.es
The possibility of knowing the accurate position of a mobile
receiver at any time using GPS technology has revolu-
tionized farming production, and in fact has led to the
development of a new concept: Precision Agriculture. The
benefits of this technological approach are numerous, and
besides the observable improvements in the quality of
regular tasks, ergonomics and safety are certainly enhanced
as intelligence is being incorporated to off-road equipment.
Laboratory models are giving way to actual applications, and
California law, for example, allows tractors in furrows
traveling less than two miles per hour to travel without a
driver, provided that the throttle, clutch, and brakes can be
controlled remotely, a practice that is wide-spread with
harvest crews [2].
There exist many channels to transmit scientific and
technological innovations, the most frequent being journal
articles, technical conferences, industry shows and patents. It
is evident that they are not mutually exclusive, and therefore
it must be an overlap among them. Consequently, even
though this paper presents a review based upon patents, an
important amount of the material covered here can also be
found in regular scientific literature. The majority of ideas
protected through patents that have been disclosed by
academic personnel, for instance, have also been published
as journal papers or conference proceedings. However, the
correspondence is not direct, and many companies only
reveal their discoveries as a need to protect their intellectual
property. For the layman, and even for engineers, the
language and format in which patents are structured may
become hermetic and difficult to grasp because expert
attorneys intend to protect the maximum while divulging the
minimum. As a result, an innovation very often results in a
family of patents where differences among individual texts
are hard to identify, and yet there are slight details making
170 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms
worth the multiplication of documents. There is a long, not
necessarily straight, route from a patent application to the
corresponding commercial product. As a matter of fact,
many patents do not finalize in concrete physical realizations
and they, generally speaking, serve as a shield for concepts
upon which further developments can be constructed without
the need of incurring in costly intellectual property expenses
due to other assignees. In spite of the scanty circulation and
trade of intellectual property documentation, since they tend
to remain within the institution holding legal rights over
them, patents stimulate innovation and technical advances.
Prog-ress spurred by industry in this field is essential and
should never be overlooked, as the successful outcomes of
the Grand Challenge competition clearly prove [3].
The objective of this article is to provide an overview of
recent innovations in automatic guidance of off-road vehicles
through the revision of key patents issued in this field.
Obviously, it is impractical to analyze every single patent
related to this novel application, but a selection of those
transmitting the fundamental knowledge involved have been
carefully examined and discussed in the following sections.
A total of sixty patent documents have been studied. Figure
(1) shows a list of the most important features included in the
set of patents considered, and the percentage of documents in
which every subject matter listed is treated. If a threshold of
popularity can arbitrarily be traced at 20%, the bar chart of
Fig. (1) indicates that the most recurrent features are: 1)
actuation through electrohydraulics; 2) safeguarding and
security mechanisms; 3) navigation via satellite localization;
4) computer vision applications; 5) control systems; and 6)
trajectory planning.
2. GENERAL SOLUTIONS FOR THE EARLY
CHALLENGES
Even though there are patents dealing with driverless
vehicles since the 1920s, substantial inventions for the
current state of the art are mainly dated from the 80s
onward. This patent review considers two different periods
of study: what took place during the 80s and 90s of the 20
th

Century; and all the innovations developed after the year
2000. The main reason for setting the division line at the turn
of the century is the boost given to this technology by a set
of circumstances such as the state of maturity reached by
vision sensors, the development of compact binocular
cameras, the cancellation of the selective availability of GPS,
the dominance of the digital imagery, etc.
Once computer power reached significant levels of
performance and sensors became available and affordable,
general solutions for autoguidance emerged. It is common
for these first inventions to cover most of the key areas in
vehicle automation. In the robot tractor envisioned by Pryor
in 1988 [4], automatic steering is achieved when a solid state
TV camera detects target boards placed in the field. A
microprocessor is able to control both steering and throttle,
generate pre-programmed 180 turns at the end of the
furrowed area, and even manipulate an arm for picking
fruits. Logically, safety has been a concern from the initial
applications, and this invention proposes a toy-like bumper















Fig. (1). Important features included in patents on automatic guidance of off-road vehicles.
Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 171
system around the tractor or a cable buried along the
boundaries of the field detectable by a magnetic sensor. In
terms of localization, the pre-GPS era could be considered
the beacons reign, as this was the preferred positioning
technique. The 1987 steering automata described in [5] was
capable of following a preprogrammed course calculating
angles and positions between the vehicle and reference
points marked by fixed beacons or rotating lasers. Steering
actuation was achieved through en electrically controlled
hydraulic valve governing the steering cylinder, method that
has remained in practice until now. When optical sensors
were not used, the alternative to global positioning was local
sensing with mechanical feelers. The mechanism of [6] was
able to follow a furrow previously traced with a gauge wheel
attached to the guided vehicle. Actuation was executed by
turning the steering wheel with an electric motor and a
transmission commanded by a rotating shaft inserted in the
leading furrow.
Before the turn of the century, general robotic systems
for assisting off-road vehicles in a wide selection of agricul-
tural operations were envisioned. The dawn of precision
agriculture was stimulating complex, and very often
imaginative, solutions to transform the traditional farm into a
production factory. The self-navigating robot proposed by
Rocks [7], for instance, claims accuracies of one inch when
performing farming tasks as tillage, planting and harvesting,
even in undulated irregular terrain. Such level of accuracy is
obtained by means of seven halogen bulb beacons which
allow for dead reckoning calculations comparable to a pre-
stored scenario loaded in the on-board computer. If an
emergency arises, the vehicle can be halted remotely. The
scouting vehicle described in [8], was developed around the
same time, but it already incorporates satellite localization to
get rid of on-site references. Both techniques coexisted
before the US Department of Defense canceled the selective
availability.
3. THE ADVENT OF SATELLITE LOCALIZATION
The real-time knowledge of a mobile machine precise
position unchained the fast development of automation in the
farm field with all kind of solutions and numberless projects
being tackled in both academy and industry. A significant
milestone in the progress of satellite navigation applications
to agriculture, where high accuracy to move between crops is
necessary, has been the deactivation of the selective
availability by a Presidential Order on 2 May, 2000 [9]. The
selective availability (SA) was an on-purpose GPS signal
degradation introduced by the Department of Defense to
control accuracy of GPS for civil use. At present, GPS is
being used extensively in rural areas where large fields allow
for efficient vehicle automation, and the debate is being
shifted from signal accuracy to in-field performance accu-
racy [10]. Even though the only completely functional
navigation satellite system is NAVSTAR GPS, several other
systems are being implemented, such as Galileo or
GLONASS, and therefore it is more appropriate to group all
of them under the common denominator GNSS (Global
Navigation Satellite System).
The applicability of GNSS to agriculture is, a priori,
boundless. All kinds of platforms can benefit from global
localization. The global positioning base station described in
[11], corrects and sends differential signals to either ground
or airborne rovers to accurately dispense agricultural
products. The ground clearing machine of [12] uses rotating
tracks with a flat-surfaced tread to better distribute weight
and improve stability. In general, GPS-guided vehicles
require the establishment of a reference, or initial line, which
many times is recalled as A-B line. The difference will be, of
course, on the steering algorithms, but the localization data is
identical. The raw data, basically time and geodesic coor-
dinates, captured in the receiver at 1 Hz or 5 Hz can be
processed in different ways in order to deduce as much
navigation information as possible. Four GPS antennas were
mounted on a tractor cabin by a team at Stanford University
[13] to determine the vehicle attitude due to roll and pitch.
This information was used to minimize the lateral position
error of the tractor. The definition of the reference line,
typically associated to the desired offset for parallel tracking,
is one of the features present in most of the satellite navi-
gation systems. This line does not need to be straight, and
curved segments are eligible as well. The trajectory to be
followed by the vehicle is usually composed of a series of
points, technically called waypoints, which draw the desired
path. The organizers of the Grand Challenge competition [3],
for instance, provide the contenders with a complete set of
waypoints from the beginning to the end line of the race. The
guidance assisting apparatus for agricultural vehicles
described in [14] is driven from one waypoint to the next
along the wayline. The wayline is a closed polygon with up
to 1000 waypoints per kilometer. When the waypoints are
not in the same plane, difficulties arise. As mentioned in
[13], a plurality of antennas can be used to estimate pitch and
roll, but this solution tends to be complex and bulky. An
alternative [15] is to integrate the GPS with a tilt measuring
device which, in addition to an electronic compass for robust
heading, forms a terrain compensation unit. In these
circumstances, common in hilly terrains, the wayline is
upgraded to a 3D line where 3D coordinates and 3D slope
angles need to be known. Generally speaking, the terrain
compensation module can comprise gyros, accelerometers,
tilt sensors and the like. The terrain compensation module is
essential to navigate inside the field, but not always the
GNSS-based guidance systems provide full autonomy to the
vehicle. An intermediate solution, known as semi-autonomy
situation, provides assistance to the operator who is actually
driving the vehicle but following the indications of the
automatic navigator [11, 16-18]. This is a fairly common
solution, mainly deployed in the first commercial units, and
is typically realized through the lightbar displays. The lateral
displacements of the vehicle are indicated by lighting an
equivalent number of lights on the proper half of a LED bar.
Lightbars are often replaced by more sophisticated monitors
where a Graphic User Interface (GUI) provides the driver
with the important data for a safe operation. The great
accuracy demands of precision agriculture applications have
resulted in the development of high performance GPS
solutions capable of localizing a receiver with sub-inch
exactitude. The method described in [18] integrates a Real-
Time-Kinematics (RTK) GPS with precision farming to
provide accurate seeding, cultivating, planting or harvesting.
Due to its accuracy, this system can make use or assist in the
construction of Geographic Information Systems (GIS) with
172 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms
the overall goal of generating a digital map including the
precise position required by each operation.
4. OPTICAL AND LOCAL PERCEPTION
After the initial euphoria brought by the easy and reason-
ably priced availability of GPS data, significant challenges
with potential severe consequences became visible and
resulted in the necessity of local perception for vehicles
operating with some degree of autonomy. Bias, drift and
guess-row errors led Rovira-Ms et al. [19] to postulate a
model for sensor fusion of machine vision and GPS to guide
a tractor at various speeds and under diverse field and sensor
signal quality conditions.
Optical range finders are universal solutions for
estimating distances as a way to perceive locally, that is, in
the vicinity of the vehicle. These sensors emit electro-
magnetic radiation and receive a reflecting beam from the
detectable object; a timer determines the elapsed time that
allows the calculation of distances. The electromagnetic
radiation can be infrared (IR), near infrared (NIR), ultra-
violet (UV), red light, the visual spectrum, etc. In a general
embodiment [20, 21], the outbound optical path consists of
the electromagnetic source, the focusing optical member
(lens), and a micro-mirror array and transmission optical
member. The inbound optical path includes the reception
optical member and a detector. A processor is in charge of
converting elapsed time into distance, and to do so it may
consider the first return, the last return, the strongest return,
the average of returns, or the median of returns. Sensing one
object can be helpful for obstacle avoidance, but guiding a
vehicle requires multiple distance measurements. The
application patented in [22], for example, utilizes a plurality
of markers located on a predetermined road; a method coined
continuous/discrete marker system. The markers are
reflective stripes painted along the path. The separation bet-
ween reflectors depends on the design and vehicle speed, but
the inventors suggest in one of the prototypes described a
spacing of 50 feet. The signal processing algorithm, calcu-
lates the heading angle with respect to the approximated path
as a mean to determine the desired steering angle of the
vehicle. The previous descriptions set the common frame-
work for local sensing, but the specific applications can be
divided into two groups: computer vision and the remainder
of range finders.
4.1. Computer Vision
The tremendous potential of machine vision as a
perception-navigation sensor for off-road vehicles, due in
part to its similitude to human visual perception, has resulted
in the coalescence of agricultural and non-agricultural
institutions pursuing similar goals regarding vehicle auto-
mation. This is the case of the self-propelled harvester
robotized by Carnegie Mellon University [23, 24]. The
harvester was endowed with a crop-tracking apparatus for
steering the vehicle along the crop without operator assis-
tance, although, for safety principles, the presence of the
operator was required in the cabin when navigating under
automatic mode. A video camera was set on a protruding
arm at 4 m height to detect the cut-uncut edge (crop line) of
the yield being harvested. A color-based discriminator func-
tion running on a microprocessor found the best position for
the crop line. The harvester executed the turns by powering
the front wheels at different speeds with hydraulic motors.
The kind of scene captured by the vision sensor shapes and
defines the image processing algorithms from which
decisions will be made. In the case just explained, the
searched feature was the crop line delimited by the cut-uncut
front, one of the most common scenarios found in agricul-
tural fields.
Another familiar in-field situation is the important case
of crops arranged in rows. For this condition, the system
proposed in [25, 26] features four regions of interest (ROI) to
identify two crop rows robustly and guide a tractor. A region
of interest is a window selected within the image where
image processing techniques are applied to extract meaning-
ful information, in our particular case about navigation. The
series of images depicted in Fig. (2) illustrates a typical
image analysis for computer vision navigation of a tractor in
a soybean field [27]. The distinction between crop and non-
crop (mainly soil) implemented in [25, 26] was based on the
k-means algorithm applied to the four regions of interest.
The k-means algorithm is a clustering technique which
minimizes the within group variance to partition the cluster,
where k represents the number of groups or classes. Once the
partitions have been realized, the algorithm applies moment
operations to determine the location and orientation of crop
rows. Linear regression is then used to define the trajectory
path. An alternative processing technique to identify the
crops and find the optimum path is based on the Hough
transform [27], applied to the image shown in the example of
Fig. (2). Machine vision techniques have been combined
with other algorithms more representative of artificial
intelligence, but really effective to cope with the usual
difficulties for an imaging device operating in open fields. A
two-class k-means algorithm fed a fuzzy logic controller to
decide the best estimate for the slope and intercept of a line
representing the cut-uncut edge detected by a video camera
mounted on a harvester [28]. Due to the fact that the camera
created a perspective shift growing with the width of the
combine head, and contrary to the case reported in [23, 24],
the camera was eventually mounted on the combine head,
making the edge an almost vertical line. In the headlands,
though, there are no cut-uncut boundaries and the farmer
needs to steer the vehicle in a semicircular path until it faces
the boundary from the opposite direction. Since direction is
reversed, the camera is too far to capture an adequate view,
and therefore a second camera was installed in the opposite
side of the corn head so that the vehicle could harvest from
both sides [29].
A third situation, in addition to the detection of har-
vesting cut-uncut lines and crop rows, is represented by the
automation of a self-propelled picker following a cut swath
of crop lying on the ground [30]. Given that the only source
of guidance features is the swath, the ROI needs to be
precisely defined to avoid the swath falling outside the
processing window. A successful technique to deal with the
swath detection challenges classified swath pixels according
to their texture information in relation to the texture of the
surroundings; the vertical stalks of grass remain standing
while the swath includes longer stalks in multiple directions,
hence the difference in texture. A grayscale dependency
Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 173
matrix based on 8-connectivity was the preferred operation
to find the swath in the input images. When the color discri-
minator of the harvester described in [23] found problems
with the Sudan grass, a texture discriminator based on the
Gabor filter was implemented instead. The localization
information deduced from image analysis can be compli-
mented with other navigation parameters such as the
odometry-based traction control on farm vehicles disclosed
in [31]. Odometry measures travel over a surface and is
normally used in dead-reckoning navigation. The accuracy
of this distance estimation may be improved with the
assistance of GPS position information. The participation of
GPS technology enhances the reliability of the measurements
because the optical odometer is a differential instrument and
accumulates errors over distance.
4.2. Other Range Finders
One of the classic alternatives to computer vision for
vehicle local perception is represented by lasers range
finders. The laser radar for lateral guidance reported in [32],
produced a pulse modulated beam and received reflected
light beams from mounted retroreflectors, covering the
portion of the road ahead of the vehicle. The distance and
angle to each reflector were calculated to determine the
position of the vehicle and to estimate the road curvature for
commanding course correction information. The road
references may be standard roadway reflectors mounted at
regular intervals of, say, 15 meters apart. A conventional
laser transmitter may be a medium power IR (100 W)
centered at a wavelength of 820-830 nm.
The other broad-spectrum local sensors for mobile
robotics are ultrasonic range finders. An ultrasonic guidance
apparatus envisioned by Coker [33] is capable of steering a
vehicle on a desired track between adjacent crop rows or
sides of a marked furrow. Onboard sensors are setup for
transmitting ultrasonic ranging signals and receiving the
reflected signals from the rows or furrows. Two ultrasonic
emitters need to point downwardly and outwardly in
opposite directions towards adjacent crop rows or plant stalk
bottoms; as the vehicle moves laterally, the distance from
one sensor will increase while the distance to the other will
decrease. A transducer measures the elapsed time of the
ultrasonic pulse to travel to the object and back to the sensor,
computing the distance based on the speed of sound. If both
transducers sense an increase of distance, the data is
disregarded because it indicates a vertical motion (bounce)
rather than a lateral displacement. This approach is, to a
certain extent, reactive and in some way reminds of mecha-
nical feelers.
5. GUIDANCE ALGORITHMS, PATH-PLANNING,
AND VEHICLE CONTROLS
The birth and development of the concepts key to
artificial intelligence has preceded the availability of fast,
efficient, and cost-effective robotic hardware, yet it has been
the fast growth of electronics and sensors what boosted
automation in off-road environments. Whatever the case may
be, any vehicle bestowed with some degree of autonomy
requires (artificial) intelligence, and the design of guidance
algorithms and control strategy is probably the central stage
in automatic navigation. This section reviews the most used
approaches to process sensor data and elaborate an actuation
plan to drive the vehicle as safe and as accurate as possible.
Different sensing techniques might make use of the same
control stratagem to steer a vehicle and vice versa; the same
localization and perception data can be interpreted in many
different ways to guide an autonomous vehicle. Being
computer vision one of the preferred techniques for vehicle
local perception, there is a rich body of vision-based algo-
rithms to resolve a vehicles trajectory. The self-propelled
harvester described in [23] couples a best fitting step
function to the harvesting frontline using the least-squared
error criterion and a vote counter. The position of the crop
line directly establishes the turning angle of the wheels,
which is executed by rotating the front wheels at different
rate. The multiple ROI guidance system for rows tracking
covered in [25, 26, 34-36] is able to trace a trajectory
expressed in a matrix with a discrete number of points, the
first five being extracted from the input image. A fixed look-
ahead time is employed to estimate look-ahead distances.
The trajectory path is determined via linear regression, and
the planner eventually finds the desired wheel angle and the
look-ahead distance along the y-axis (forward direction). The
same k-means algorithm but with k =2 [28] was used to
track the cut-uncut edge of corn being harvested. The
separation line was determined by regression analysis and
the acceptability of the solution evaluated through fuzzy
logic membership functions: a perfect vertical line (infinite
slope) indicates that the camera is located over the cut-uncut
edge [29]. A PID steering controller governs the hydraulic
cylinders of the steering mechanism in the rear wheels of the
combine. Commands from the controller excite a solenoid
that actuates a proportional control valve.




(a) (b) (c) (d)
Fig. (2). Typical image analysis for navigation of off-road vehicles: (a) Region of interest selection; (b) Segmentation; (c) Mid-point
encoding; and (d) Row identification and optimal path [27].
174 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms
Vehicle navigation entails the definition of the desired
destination to be reached by the automatic machine, and this
can range from a simplistic reactive system such as
mechanical feelers to a sophisticated trajectory planner. An
example of an intermediate case is represented by the lateral
guidance control system described in [37], where the
calculation of the target point is based on the length of the
preview distance obtained as the intersection of the radius of
half circle with road geometry. Steering angles are computed
with linear equations as functions of marker gains and
heading angles. The multisensory harvester equipped with
GPS, INS, and two video cameras of [38] followed a field
coverage plan generated by a trajectory planner. The targeted
speeds are about 4 or 5 miles per hour (6-8 km/h) and the
main controller follows a behavior-based approach. This
control strategy was conceived by Rodney Brooks [39] and
establishes a direct link between perception and actuation,
circumventing cognition by high-level reasoning entities.
Conventional wisdom establishes that before reaching the
state of full autonomy, there will be many ideas,
applications, prototypes, and solutions regarding semi-
autonomy. The majority of the works reported so far request
the presence of drivers at the extremes of the fields to
perform turns between passes, and a good policy for
engaging and disengaging automatic control becomes a
matter of efficiency. The method developed in [40]
initializes automatic guidance after turning at the end of the
path. When the operator moves the steering mechanism,
autoguidance deactivates to allow the driver perform the
turning maneuver; after it, the position of the steering
mechanism is monitored, and if it falls within a
predetermined zone, autoguidance remains activated. Other-
wise, it is disconnected. This system lets automatic control
remain connected when the machine is turning and the
steering mechanism is not in the straight travel position,
regarded as the deadband zone. The deadband zone encom-
passes several degrees on either side to account for small
variations in steering. Once automatic control is engaged,
most of the systems rely on fluid power to control the
vehicle. The harsh and force-demanding nature of off-road
equipment duties places hydraulics as the preferred actuation
mode. The vehicle mentioned in [14] features a PID
controller to move from one waypoint to the next, getting
feedback from a wheel angle sensor. The robotized tractor of
[41] possesses various sensors whose data is fused for real-
time guidance. It combines map-based guidance and sensor-
based guidance because no individual sensing technology is
ideally suited for vehicle automation. Data fusion is carried
out with the Kalman filter. The actuator is, again, an electro-
hydraulic valve commanded with Pulse Width Modulation
(PWM) signals. The wheel angle sensor is a rotary
potentiometer aligned with the king pin on the right wheel.
The basic difference between semi-autonomous and auto-
nomous guidance rests on actuation; the rest of the process
remains the same. Therefore, both systems may include
similar localization and perception systems, identical control
algorithms and trajectory planners, but only the latter is
expected to implement actuating mechanisms to guide the
vehicle, typically through hydraulic cylinders. A popular
semi-automatic solution is represented by the so-called
assistive systems. The embodiment of one of such systems
[42, 43] utilizes a feedback control system to compare the
actual position of the vehicle with the desired path, providing
feedback to the human operator through a visual display. The
enhancement in efficiency conducting regular farming tasks
is considerable with assistive systems. The complexity of the
control algorithm is comparable to those implemented in
autonomous navigators, and the assistive system mentioned
above, for instance, proposes a third order scheme (three
poles) for under-damped response with variable (velocity
dependent) gains k
p
, k
s
, and k
h
.
Very often, the output data from localization or percep-
tion engines directly feed the vehicle controller to execute
the essential functions of steering, braking, or accelerating.
However, some situations and complex solutions insert a
trajectory planner in-between. One non-trivial scenario is
found when a vehicle is guided along adjacent swaths across
terrain with slope [44]. The solution can be outlined in three
consecutive steps: first, estimate the tilt of the vehicle due to
the terrain slope; second, use GPS to find the position of a
fixed point on earth; and third, from tilt and position
measurements, calculate guiding commands to prevent gaps
and overlaps. A complete solution can be understood as the
one comprising curved and straight segments, headland
turns, and a plurality of modes. The vehicle foreseen in [45]
accepts three operation modes (manual, semi-autonomous,
and autonomous), projects a plurality of trajectories (parallel
tracking or several row skipping patterns), and adapts the
forward velocity to the position of the vehicle in the field
(turn-entry velocity, turn velocity, and turn-exit velocity).
The region-filling module (path planner) weaves the path
pattern within the perimeter defined by the user. The A
*

algorithm is the internal engine in charge of producing the
interconnected segments that form the preferred path [46]. In
the embodiment of this complete solution, the path planner
actually comprises a perimeter training module which
collects location data and a contour region-filling module
[47]. The starting input data for the planner is the vehicle
minimum turning radius, the task width (note that vehicular
width may vary from implement width), and the reference
row location. The interference module establishes geometric
constraints between adjacent arcs to avoid interferences, that
is to say, when two contiguous components converge and
there is no room to fit both. The reference contour must be
chosen in such a way that the energy consumption for
completion of coverage is minimized. The reference path is
defined by the coordinates and heading of the starting and
final points; once known, the planner identifies candidate
paths. An estimator calculates an economic cost for traver-
sing each candidate path. The economic cost depends on
distance and duration. The estimator may eliminate a path if
it is not drivable, collides with obstacles, or the economic
cost is above a determined threshold. By applying these
rules, the estimator is capable of eliminating unnecessary
looping [48]. For certain regions, spiral paths may be more
efficient than linear or other paths in terms of time, fuel, or
the like. In order to cope with these situations, a spiral
region-filling module is designed [49] with the potential to
map out contour rows following a spiral, plan a path
comprising nested loops, or generate a curve of variable
radius. Spiral shapes might have problems with some work
areas, such as those with pinch spots, and require a partial
Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 175
application. If the work area is mostly circular, an Archi-
medean spiral may be used. When contours run adjacent to a
long side of the region, the spiral tends to minimize flip turns
to reduce time. When the desired path is defined by a set of
waypoints, and the turning radius of the vehicle can imperil
the completion of the mission, sophisticated courses need to
be taken such as the bi-variate periodic bi-cubic -spline
fitting curve, which is known to yield a smooth curve
between waypoints [50]. With complex geometry, such as
spirals and loops, it might be confusing to separate the
regular tracking course from headland turns. The method
developed in [16] specifically defines turns as changes in
heading 30 over a distance of 100 m.
6. PRECISION AGRICULTURE AND IMPLEMENT
CONTROL
It seems somewhat contradictory to increase accuracy in
a vehicles localization system, by means of RTK-GPS for
example, while incurring in gross errors controlling the
implements as a consequence of key sensors being mounted
on the vehicle and not on the tool executing the task. This
problem becomes more severe as machinery grows.
Incidentally, outsized terrains are the most benefitted from
precision farming applications, but on the other hand, they
usually require large-scale equipment. For this reason,
implement control is becoming a recurrent topic in this novel
farming technique. Precision agriculture is, from and by its
genesis, inseparable from global satellite localization.
Furthermore, very frequently it combines real-time GNSS
information with GIS and aerial photographs of farms to
create rich digital maps. A generic example of a site-specific
farming system is given in [51], where the boundaries of
agricultural fields are identified to execute functions depen-
ding on the position of the vehicle: variable-rate application
and implement height depends on boundary information. The
speed of the vehicle can be adjusted according to the
distance of the vehicle to the boundaries. Implement ele-
vation, as it will be in most of the applications described
below, is controlled with relays, solenoids, and PWM gene-
rators.
Information technology (IT) has enriched the corpus of
data handled by intelligent machines, and consequently
autoguidance is, far from an isolated and curious feature as
in the beginning, another characteristic of a site model of the
field, where a path planning system is adapted to the model,
the implement, and the operation [52]. The site model
manages parameters such as yield, rainfall, prescriptions
(rate, implement elevation, and application speed), boundary
information, and obstacles data. During normal operation,
the vehicles also stores data for the site model of the field,
which is allocated in the site database, the operations
database, or the implement database. Nevertheless, precise or
site-specific applications can manage without satellite
positioning in especially controlled atmospheres. Let us con-
sider the case of a self-navigating robotic vehicle enclosed in
a garden with undulated irregular terrain [53]. The vehicle
can follow a specified course by dead-reckoning if multiple
navigation beacons are placed all around the perimeter of the
garden. The navigating module can estimate position and
orientation with accuracies of one inch by just processing the
images captured with a camera detecting the beams from the
beacons. The robot controls work implements without
human intervention according to a pre-stored scenario called
the work order. The work order can include route, speed,
turning radii, implement control points, etc. This mode of
establishing a tasking protocol for the vehicle applies
industrialization approaches to agricultural production. The
invention proposed by Hanley [54] applies flexible
manufacturing software, robotics, and sensing techniques to
agricultural operations. The idea is to move programmable
workstations typically used in manufacturing operations to
farm fields through mobile robots that can sense their
environment and respond with the typical flexibility of
manufacturing processes. The specific tasks considered are
pruning, harvesting, weeding, trimming, planting, spraying,
pollinating, or any operation on plants comprising a mobile
robotic machine with an arm and a decision making unit
based on artificial intelligence. In this approach, a human
operator is essential either locally or remotely, with some
autonomy also available. Captivatingly enough, it also points
at the convenience of this application to specialty crops, a
production niche that every day is demanding more robotic
solutions to lower labor costs while augmenting selectivity
and adaptability, or in other words, intelligence.
Aerial photographs and GIS-generated maps cannot catch
up with the dynamism involved in the active production of
farms run by precision agriculture principles. The ideal
situation is mapping while working in such a way that the set
of spatial variability maps are always being updated in real
time. This is the case of [55], where a supporting member
with a plurality of sensors is set to gather perception
information in combination with the vehicles position to
create a map of the field, as the presence or absence of weeds
at specific locations to build a weed-map. The mapping unit
is connected to the control unit and a display via CAN
protocol, so that speed and optical sensors are able to open
and close spray nozzles at the right time. In addition to
regulating chemicals or irrigation water, height control in
implements is crucial for automating farming operations.
The control circuit designed in [56] generates signals to raise
and lower agricultural tools such as plows and combine
heads. A height sensor is coupled to the control circuit to
sense the implements height and therefore provide the
feedback signals required by a closed loop control. This
automatic control decreases operator workload while increa-
sing accuracy in comparison to manual control. Obviously,
the higher precision in controlling the implement, the higher
accuracy in the operation, therefore it makes sense to invest
in improving sensing and actuating technology as a means to
enhance productivity. One way of reinforcing localization
quality is by coupling two positioning receivers with a
power-actuated hitch to maintain an implement on course
[57]. The hydraulic cylinder of the hitch is equipped with a
potentiometer for position feedback. Another way to melio-
rate the performance of automated tasks is by upgrading the
positioning technology. The system reported in [58]
integrates RTK-GPS with precision farming to provide
accurate seeding, cultivating, planting, or harvesting. A
seeder can be configured as a boom supporting one or more
GPS antennas. If the antennas are placed at either end of the
boom, the elevation of each end can be estimated. Precise
seeding with RTK-GPS may provide means for micro-crop
176 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms
rotation: rotation of the position of crop plants within a small
area of the field, considering cm-level accuracy. The
ultimate step in reaching high accuracy with the implement
actuation is by mounting the GPS antenna (RTK if feasible)
directly on the farm implement, and coupling the antenna to
the receiver [59, 60]. The implement has the tendency to
displace to the side opposite the direction that the tractor is
steered. This implement guidance system provides a fast
response for dynamically adjusting side-to-side positioning
of the implement. The automatic hitch includes an actuator
and a position sensor. The actuators are dual hydraulic
devices for pushing and pulling the tongue of the hitch. The
control computer reads the geographic information of the
path from memory and determines the cross track error of
the implement rather than the vehicle. The authors claim that
this farming system corrects the location of the implement in
less than one half the time or distance required with ordinary
on-vehicle guidance. The computer handles two parallel
operations: vehicle guidance and implement guidance. An
alternative, non-GPS related, attention-grabbing control of a
guided implement with extending arms for spraying the
ground is illustrated in [61]. This technique uses fluorescent
chemicals to mark regions, to be exact 4-6 inches wide
bands, and determine the position of the implement to keep it
on course. The objective is to avoid overlapping and
underlapping of chemicals. The spraying boom requires a
nozzle for depositing a band of fluorescent chemical, which
in turns will be excited by a light source. A hydraulic motor
governed by a proportional valve brings the implement on
track. This system can control large implements with booms
over 120 feet.
7. SYSTEM RELIABILITY AND SAFEGUARDING
Safeguarding is probably the factor that has delayed the
desired transformation from concept to product the most.
Rural environments, where heavy and powerful machines
regularly operate, are potential candidates for mortal
accidents. Leading manufacturers do not want to have their
reputation damaged and consequently tend to be conser-
vative in the implementation and commercialization of new
technology, mainly with regards to vehicle automation.
Errors accepted to drivers are not accepted to machines
which usually need to outperform skilled operators.
The wide scope of onboard safety systems ranges from
basic semi-autonomous applications to sophisticated robotic
solutions. The invention covered in [62] is designed to aid
humans in the safe use of equipment by deploying a messag-
ing network and monitoring remote control operations. The
system includes a sensory device for monitoring and
reporting data, a memory device, at least one processor, and
a plurality of external devices such as pagers and cell phones
to communicate via security protocol with encryption
technology. A probable hazard for an automatically guided
vehicle performing a task in the field is the presence of an
unexpected obstacle interfering with its trajectory. The
natural reaction for the vehicle would be to detect the
obstacle and move around it if there is enough space. In
order to do so, the controller must contain an obstacle
avoidance algorithm. The point-to-point path planning of
[48] is capable of detecting obstacles and defining a
boundary zone about each obstacle. The obstacle clearance
zone is defined by modeling a perimeter as a percentage-
augmented periphery and is based on such vehicle cons-
traints as width, wheelbase, and turning radius. The planner
identifies candidate paths and eliminates those colliding with
obstacles. The number of touches of the obstacle zones is
limited. Another method for getting information about
objects in an environment around a vehicle uses infrared (IR)
light to create a 3D representation of the surroundings based
on the measured distances to the objects [63]. The security
system can be amplified with the addition of an ultrasonic
system in an optical-acoustical combination. Dangerous
situations are warned by turning on the hazard lights, soun-
ding the horn, or similar actions. A remarkable advantage of
this system is that even in the presence of strong sunlight, a
narrow beam of IR laser can be observed if using a narrow
notch frequency filter on the received light. This embo-
diment is recommended for air seeding systems, precision
planters, and sprayers. When the vehicle approximates too
much to an interfering obstacle, stronger warning systems
need to be activated. The anticipatory sensor system of [64]
informs that an impact involving the vehicle is about to
occur unless a corrective action is taken. This action is based
on data obtained prior to the impact. Potential applications of
the system include protection to pedestrians, lane departure
warning signaling, and night driving assistance. Lasers,
cameras, GPS, and IMU are some of the sensors employed to
presage an impact. In its adaptation to agricultural vehicles
[65], a ground speed sensor is mounted onboard to calculate
wheel slip of the tractor or sprayer. The obstacle detection
algorithm utilizes pattern recognition techniques to identify
the dangerous objects [66], and some of the habitual
processing methods are neural networks, Fourier analysis, or
fuzzy logic. In addition to reactive trajectory re-planning
induced by local perception sensors, when global positioning
is part of the navigation system, reliability can also be
improved by tracking a diagnostic function to ensure that the
navigation system is working properly [14]. This function
can include the number of satellites, quality indices such as
de dilution of precision, CPU load, or even the solenoid
response.
8. CURRENT & FUTURE DEVELOPMENTS
In spite of more than two decades of intense and fertile
development of automatic guidance technology, we can
consider this technology a young technology from the
customer acceptance standpoint; if it is true that some early
adopters are regularly using some of the few commercial
systems available in the market, the majority of professional
producers still remain stuck to conventional systems and
procedures. There is still a long way to go, although users
grow year after year and the trend seems to be favorable. The
incorporation of new satellite systems to the arena of GNSS,
such as GLONASS and Galileo, will probably lead to a
wider use, perhaps universal, in agriculture, forestry, and
construction. The side effect will certainly be the increase in
complexity of receivers which will need to accept different
sources of signals, but on the other hand, reliability will also
be enhanced as redundant signals result in safer outcomes.
This GNSS augmentation will have direct repercussions on
precision agriculture future paths as spatial variability will be
better tracked with the new localization data abundance.
Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 177
Western countries seem to find troubles with conventional
agriculture while developing countries need technology for a
sustainable production; precision farming appears as a
factual solution to assist both needs. The incorporation of
these technological advances, however, will require several
steps. Hardware and software reliability needs to be
guaranteed for the harsh environments found in agricultural
fields. Extension personnel should smooth the transfer from
research laboratories to farming sites. Manufacturers, in
conjunction with dealerships, have to be prepared to instruct
and assist their customers in the operating techniques and
data handling.
The analysis of the technological evolution around
intelligent vehicles can bring some insights to predict what
might occur in the next decade. The bar chart of Fig. (3)
represents the weight of key features found in automatic
guidance patents according to their period of publication:
1985-2000 and 2001-2008. It is apparent in the graphic that
beacons, pseudolites, and non-vision local sensors have
experienced a recession in favor of computer vision systems
and algorithms, probably motivated by the significant growth
in computer power. Inventions related to precision agri-
culture were more numerous in the first period, which is
surprising because selective availability disappeared after
2000. It may be possible, though, that the loss in GPS signals
restrictions has weakened research devoted to compensate
for signal degradation. Nevertheless, the applications of GPS
systems, as expected, have increased significantly, with 80%
of the patents citing this technology being published after
2000, and 90% of the inventions dealing with parallel
tracking also belonging to the period 2001-2008. It is note-
worthy the fact that most of the semi-autonomous appli-
cations reported in the reviewed patents have been developed
in the second period, and lightbars guiding assistance is
dominant in the second period as well. After the oneiric start
of vehicle automation where some people envisaged farms of
crowded machines multitasking on their own, roboticists and
engineers found a modest commencement based on the
deployment of semi-autonomous equipment a more reason-
able approach for consolidating this technology. Guidance
visual assistance through lightbar displays and straight-alone
autopilots turned out to be grounded solutions to begin with.
Headland turns and field-filling complete trajectories are in
the agenda but still need the support of highly safe software
and hardware. In light of these evidences, the coming years
will probably see a continuation of GNSS applications with
better signals and filters, a perfection of imaging sensors, and
the expected enhancement of reliability and safety. Sensor
fusion of global and local information might also step up.
The Grand Challenge competition demonstrated that the
possibility of endowing a vehicle with advanced intelligence
is realistic; the required technology is available and reason-
ably reachable. Yet the competition was a giant laboratory
where high-tech prototypes were defied, and it will take
some time for manufacturers to incorporate automatic gui-
dance kits in their mass-produced models. Some of the initial
problems such as sensor accuracy, processing speed, or cost
are being fixed with time; but reliability is still an unsolved















Fig. (3). Technological evolution of auto-guidance for the periods 1985-2000 and 2001-2008.
178 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms
problem. For this reason, perceptual devices will expand
their horizons, and more sophisticated sensors as stereo-
scopic vision heads or 3D laser scanners, perhaps borrowed
from space technology, will join the already numerous
family of hardware integrated in intelligent vehicles. In any
case, the essence of a vehicles behavior, its intelligence after
all, will be dictated by the onboard algorithms and program-
med conducts, and from that stand-point, significant advan-
ces are expected in the philosophical conception of the
vehicles brain.
ACKNOWLEDGEMENTS
I would like to express my most earnest gratitude to J ohn
Reid, Qin Zhang and Shufeng Han for being the individuals
who introduced me into this fascinating field, taught me the
essentials, and motivated me to continue within its boun-
daries and even transcend them.
CONFLICT OF INTEREST
None known.
REFERENCES
[1] Reid J F, Zhang Q, Noguchi N, Dickson M. Agricultural automatic
guidance research in North America. Comput Electron Agr 2000;
25: 155-167.
[2] Shirley R. Ventura Countys Changing Agriculture. Resource;
February 2008: 5-8.
[3] Gibbs WW. Innovations from a robot rally. Sci Am 2006; 294(1):
64-71.
[4] Pryor, T.R.: US4769700 (1988).
[5] Dyke, H.L.: US4700301 (1987).
[6] Barnes, R.L., Gary, D.A.: US5121799 (1992).
[7] Rocks, J .K.: US5974348 (1999).
[8] Hale, G.H., Wendte, K.W.: US5870689 (1999).
[9] Misra P, Enge P. Global Positioning System. Signals, Measure-
ments, and Performance. Ganga-J amuna Press: Lincoln, Mass, USA
2006.
[10] Rovira-Ms F, Han S, Reid J F. Trajectory-based algorithm for the
quantitative evaluation of automatically steered vehicles. Trans
ASABE 2008; 51(5): 1545-1552.
[11] Goodwin, J .L., Hartt, J .R., Vetter, A.A., Walton, J .T., Bletzacker,
F.R., Forgette, T.M., Hughes, A., Neuperger, F.H., Falkenberg,
W.H.: WO9515499 (1995).
[12] Mullins, D. B.: US6377881 (2002).
[13] Parkinson, B.W., O'Connor, M.L., Elkaim, G.H., Bell, T.:
US20006052647 (2000).
[14] Mailer, R.L.: US6876920 (2005).
[15] Nichols, M.E.: US6501422 (2002).
[16] Lindores, R.J .: US7363154 (2008).
[17] Lindores, R.J .: US20070083299 (2007).
[18] Keller, R.J ., Nichols, M.E., Lange, A.F.: US6199000 (2001).
[19] Rovira-Ms F, Han S, Wei J , Reid J F. Fuzzy logic model for sensor
fusion of machine vision and GPS in autonomous navigation.
ASABE Annual Meeting 2005: Paper No. 051156.
[20] Anderson, N.W., Thackray, R.G.: US7064810 (2006).
[21] Anderson, N.W., Thackray, R.G.: US7206063 (2007).
[22] Margolis, D.L., Yasui, Y.: US5357432 (1994).
[23] Stentz, A., Hoffman, R., Ollis, M., Whittaker, W., Fitzpatrick, K.:
EP0801885 (2002).
[24] Stentz, A., Ollis, M., Fitzpatrick, K., Hoffman, R., Whittaker, W.:
US5911669 (1999).
[25] Dickson, M.A., Ni, B., Han, S.: US6285930 (2001).
[26] Dickson, M.A., Ni, B., Han, S., Reid, J .F.: US6385515 (2002).
[27] Rovira-Ms F, Zhang Q, Reid J F, Will J D. Hough-transform-based
vision algorithm for crop row detection of an automated agricultural
vehicle. J Automobile Eng 2005; 219: 999-1010.
[28] Benson, E.R., Zhang, Q., Reid, J .F.: US6721453 (2004).
[29] Benson, E.R., Zhang, Q., Reid, J .F.: US6819780 (2004).
[30] Hofer, J ., Bohner, M., Freichel, T.: US7400957 (2008).
[31] Weinstein, L., Sinclair, K., Gainsboro, J ., Willison, P.:
WO2005084155 (2005).
[32] Frank, A.A., Nakamura, M.: US5202742 (1993).
[33] Coker, W.B.: US5410479 (1995).
[34] Dickson, M.A., Ni, B., Han, S.: US6686951 (2004).
[35] Dickson, M.A., Ni, B., Han, S.: US6278918 (2001).
[36] Dickson, M.A., Ni, B., Han, S.: US6490539 (2002).
[37] Margolis, D., Yasui, Y.: US5390118 (1995).
[38] Pangels, H., Pilarski, T., Fitzpatrick, K., Happold, M., Ollis, M.,
Whittaker, W., Stentz, A.: US6336051 (2002).
[39] Brooks RA. Cambrian Intelligence. The Early History of the New
AI. The MIT Press: Cambridge 1999.
[40] Motz, D.S., Saxsma, A.L., Staub, M.D.: US6738695 (2004).
[41] Dickson, M.A., Noguchi, N., Zhang, Q., Reid, J .F., Will, J .D.:
US6445983 (2002).
[42] Harrison, J r. J .H., Harrison, H.J .: US6925380 (2005).
[43] Harrison, J r. J .H., Harrison, H.J .: US20060089796 (2006).
[44] Nichols, M.E.: US6703973 (2004).
[45] Gray, S.A., Hansen, S.L., Flann, N.S.: US6907336 (2005).
[46] Gray, S.A., Hansen, S.L., Flann, N.S.: US7110881 (2006).
[47] Flann, N.S., Hansen, S.L., Gray, S.A.: US7216033 (2007).
[48] Flann, N.S., Hansen, S.L., Gray, S.A.: US7079943 (2006).
[49] Flann, N.S., Hansen, S.L., Gray, S.A.: US7228214 (2007).
[50] Overschie, J .: US7277792 (2007).
[51] Hale, G.H., Wendte, K.W.: US5978723 (1999).
[52] Motz, D.S., Staub, M.D.: US6236924 (2001).
[53] Rocks, J .K.: US5974348 (1999).
[54] Hanley, B.P.: US6671582 (2003).
[55] Beck, J .L., Kinter, M.L.: US5809440 (1998).
[56] Hale, G.H., Wendte, K.W., Orbach, A.: US5961573 (1999).
[57] Mcclure, J .A., Heiniger, R.W., Timm, J .T.E., Funk, K.D., Wong,
R.B.: US7373231 (2008).
[58] Keller, R.J ., Nichols, M.E., Lange, A.F.: US6199000 (2001).
[59] Lange, A.F., Funk, T.J .: US7054731 (2006).
[60] Lange, A.F., Funk, T.J .: US7383114 (2008).
[61] Sauter, G.F., Nelson, G.F., Andersen, D.P., Paulinski, D.W.:
US5636792 (1997).
[62] Walker, R.C.: WO01054044 (2001).
[63] Breed, D.S., DuVall, W.E., J ohnson, W.C.: US7209221 (2007).
[64] Breed, D.S.: US20080040004 (2008).
[65] Breed, D.S.: US20050278098 (2005).
[66] Breed, D.S., DuVall, W.E., J ohnson, W.C.: US20070182528
(2007).

S-ar putea să vă placă și