Recent Innovations in Off-Road Intelligent Vehicles: In-Field Automatic Navigation Francisco Rovira-Ms* School of Agricultural Engineering, Polytechnic University of Valencia, Valencia, Spain Received: March 25, 2009; Accepted: May 27, 2009; Revised: May 29, 2009 Abstract: The fast evolution of computing power together with the improvement of electronics and sensors in the last decade has resulted in the introduction of vehicle automation at a consumer level; therefore, every day, an increasing amount of commercial solutions are reaching the global market. The special shape in which agricultural fields are structured, on the other hand, makes off-road vehicles one of the best candidates for driving assistance. Very often, tractor or harvester operators are required to follow predefined rows for long periods of time, leading to physical and mental fatigue that can provoke dangerous situations or even fatal accidents. In such conditions, the aid brought by intelligent systems can represent a positive asset to farm production systems. The recent popularization of satellite navigation systems, such as GPS, has meant the final thrust to the deployment of automatic guidance systems. However, not only global positioning sensors are key in vehicle automation but a complex architecture comprising imaging, laser and ultrasonic devices can be found in recent models as well. This article presents an overview of patents issued in this field of automotive engineering over the last twenty years. Keywords: Automatic guidance, off-road vehicles, GPS auto-steering, farm machinery automation, autonomous navigation, field robotics, agricultural robotics, crop-tracking vehicles, tractor guidance & intelligent vehicles. 1. BACKGROUND AND SCOPE Automatic guidance of vehicles is one of those themes where imagination and popular imagery has long preceded technology. As early as 1924, Willrodt described a system to guide a machine following furrows across a field, and interestingly enough, it was published as a patent [1]. Since the time of that pioneering work, engineers have been lucu- brating and imagining ways to make vehicles move without a driver. However, the solutions found were quite limited and specifically adapted to particular environments. The silicon revolution and digital era of the sixties and seventies was going to change the scene; the fast growth of technology in computers and electronics soon hinted that the old dream of autonomous robots might be feasible. Yet there was a long way to go, but the special know-how that automatic navi- gation requires was being developed at a high pace. Never- theless, not until the eighties did ideas start to take form with the first prototypes beginning the desired trip from concept development to product development. The definite stroke has been banged by the popularization of the Global Positioning System (GPS), which allows a real-time localization of a mobile platform in a global frame. With all the ingredients for automation obtainable, it makes sense to wonder if there is still a strong interest in the automation of off-road vehi- cles. The answer is affirmative, and the increasing number of patents and scientific articles focusing on these topics demonstrates that they are of intense and immediate interest. Spatial variability is a typical phenomenon in agriculture which traditional farming systems find difficult to cope with.
*Address Correspondence to this author at the School of Agricultural Engineering, Polytechnic University of Valencia, Camino de Vera, s/n, 46022 Valencia, Spain; Tel: +34-963877291; Fax: +34-963877299; E-mail: frovira@dmta.upv.es The possibility of knowing the accurate position of a mobile receiver at any time using GPS technology has revolu- tionized farming production, and in fact has led to the development of a new concept: Precision Agriculture. The benefits of this technological approach are numerous, and besides the observable improvements in the quality of regular tasks, ergonomics and safety are certainly enhanced as intelligence is being incorporated to off-road equipment. Laboratory models are giving way to actual applications, and California law, for example, allows tractors in furrows traveling less than two miles per hour to travel without a driver, provided that the throttle, clutch, and brakes can be controlled remotely, a practice that is wide-spread with harvest crews [2]. There exist many channels to transmit scientific and technological innovations, the most frequent being journal articles, technical conferences, industry shows and patents. It is evident that they are not mutually exclusive, and therefore it must be an overlap among them. Consequently, even though this paper presents a review based upon patents, an important amount of the material covered here can also be found in regular scientific literature. The majority of ideas protected through patents that have been disclosed by academic personnel, for instance, have also been published as journal papers or conference proceedings. However, the correspondence is not direct, and many companies only reveal their discoveries as a need to protect their intellectual property. For the layman, and even for engineers, the language and format in which patents are structured may become hermetic and difficult to grasp because expert attorneys intend to protect the maximum while divulging the minimum. As a result, an innovation very often results in a family of patents where differences among individual texts are hard to identify, and yet there are slight details making 170 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms worth the multiplication of documents. There is a long, not necessarily straight, route from a patent application to the corresponding commercial product. As a matter of fact, many patents do not finalize in concrete physical realizations and they, generally speaking, serve as a shield for concepts upon which further developments can be constructed without the need of incurring in costly intellectual property expenses due to other assignees. In spite of the scanty circulation and trade of intellectual property documentation, since they tend to remain within the institution holding legal rights over them, patents stimulate innovation and technical advances. Prog-ress spurred by industry in this field is essential and should never be overlooked, as the successful outcomes of the Grand Challenge competition clearly prove [3]. The objective of this article is to provide an overview of recent innovations in automatic guidance of off-road vehicles through the revision of key patents issued in this field. Obviously, it is impractical to analyze every single patent related to this novel application, but a selection of those transmitting the fundamental knowledge involved have been carefully examined and discussed in the following sections. A total of sixty patent documents have been studied. Figure (1) shows a list of the most important features included in the set of patents considered, and the percentage of documents in which every subject matter listed is treated. If a threshold of popularity can arbitrarily be traced at 20%, the bar chart of Fig. (1) indicates that the most recurrent features are: 1) actuation through electrohydraulics; 2) safeguarding and security mechanisms; 3) navigation via satellite localization; 4) computer vision applications; 5) control systems; and 6) trajectory planning. 2. GENERAL SOLUTIONS FOR THE EARLY CHALLENGES Even though there are patents dealing with driverless vehicles since the 1920s, substantial inventions for the current state of the art are mainly dated from the 80s onward. This patent review considers two different periods of study: what took place during the 80s and 90s of the 20 th
Century; and all the innovations developed after the year 2000. The main reason for setting the division line at the turn of the century is the boost given to this technology by a set of circumstances such as the state of maturity reached by vision sensors, the development of compact binocular cameras, the cancellation of the selective availability of GPS, the dominance of the digital imagery, etc. Once computer power reached significant levels of performance and sensors became available and affordable, general solutions for autoguidance emerged. It is common for these first inventions to cover most of the key areas in vehicle automation. In the robot tractor envisioned by Pryor in 1988 [4], automatic steering is achieved when a solid state TV camera detects target boards placed in the field. A microprocessor is able to control both steering and throttle, generate pre-programmed 180 turns at the end of the furrowed area, and even manipulate an arm for picking fruits. Logically, safety has been a concern from the initial applications, and this invention proposes a toy-like bumper
Fig. (1). Important features included in patents on automatic guidance of off-road vehicles. Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 171 system around the tractor or a cable buried along the boundaries of the field detectable by a magnetic sensor. In terms of localization, the pre-GPS era could be considered the beacons reign, as this was the preferred positioning technique. The 1987 steering automata described in [5] was capable of following a preprogrammed course calculating angles and positions between the vehicle and reference points marked by fixed beacons or rotating lasers. Steering actuation was achieved through en electrically controlled hydraulic valve governing the steering cylinder, method that has remained in practice until now. When optical sensors were not used, the alternative to global positioning was local sensing with mechanical feelers. The mechanism of [6] was able to follow a furrow previously traced with a gauge wheel attached to the guided vehicle. Actuation was executed by turning the steering wheel with an electric motor and a transmission commanded by a rotating shaft inserted in the leading furrow. Before the turn of the century, general robotic systems for assisting off-road vehicles in a wide selection of agricul- tural operations were envisioned. The dawn of precision agriculture was stimulating complex, and very often imaginative, solutions to transform the traditional farm into a production factory. The self-navigating robot proposed by Rocks [7], for instance, claims accuracies of one inch when performing farming tasks as tillage, planting and harvesting, even in undulated irregular terrain. Such level of accuracy is obtained by means of seven halogen bulb beacons which allow for dead reckoning calculations comparable to a pre- stored scenario loaded in the on-board computer. If an emergency arises, the vehicle can be halted remotely. The scouting vehicle described in [8], was developed around the same time, but it already incorporates satellite localization to get rid of on-site references. Both techniques coexisted before the US Department of Defense canceled the selective availability. 3. THE ADVENT OF SATELLITE LOCALIZATION The real-time knowledge of a mobile machine precise position unchained the fast development of automation in the farm field with all kind of solutions and numberless projects being tackled in both academy and industry. A significant milestone in the progress of satellite navigation applications to agriculture, where high accuracy to move between crops is necessary, has been the deactivation of the selective availability by a Presidential Order on 2 May, 2000 [9]. The selective availability (SA) was an on-purpose GPS signal degradation introduced by the Department of Defense to control accuracy of GPS for civil use. At present, GPS is being used extensively in rural areas where large fields allow for efficient vehicle automation, and the debate is being shifted from signal accuracy to in-field performance accu- racy [10]. Even though the only completely functional navigation satellite system is NAVSTAR GPS, several other systems are being implemented, such as Galileo or GLONASS, and therefore it is more appropriate to group all of them under the common denominator GNSS (Global Navigation Satellite System). The applicability of GNSS to agriculture is, a priori, boundless. All kinds of platforms can benefit from global localization. The global positioning base station described in [11], corrects and sends differential signals to either ground or airborne rovers to accurately dispense agricultural products. The ground clearing machine of [12] uses rotating tracks with a flat-surfaced tread to better distribute weight and improve stability. In general, GPS-guided vehicles require the establishment of a reference, or initial line, which many times is recalled as A-B line. The difference will be, of course, on the steering algorithms, but the localization data is identical. The raw data, basically time and geodesic coor- dinates, captured in the receiver at 1 Hz or 5 Hz can be processed in different ways in order to deduce as much navigation information as possible. Four GPS antennas were mounted on a tractor cabin by a team at Stanford University [13] to determine the vehicle attitude due to roll and pitch. This information was used to minimize the lateral position error of the tractor. The definition of the reference line, typically associated to the desired offset for parallel tracking, is one of the features present in most of the satellite navi- gation systems. This line does not need to be straight, and curved segments are eligible as well. The trajectory to be followed by the vehicle is usually composed of a series of points, technically called waypoints, which draw the desired path. The organizers of the Grand Challenge competition [3], for instance, provide the contenders with a complete set of waypoints from the beginning to the end line of the race. The guidance assisting apparatus for agricultural vehicles described in [14] is driven from one waypoint to the next along the wayline. The wayline is a closed polygon with up to 1000 waypoints per kilometer. When the waypoints are not in the same plane, difficulties arise. As mentioned in [13], a plurality of antennas can be used to estimate pitch and roll, but this solution tends to be complex and bulky. An alternative [15] is to integrate the GPS with a tilt measuring device which, in addition to an electronic compass for robust heading, forms a terrain compensation unit. In these circumstances, common in hilly terrains, the wayline is upgraded to a 3D line where 3D coordinates and 3D slope angles need to be known. Generally speaking, the terrain compensation module can comprise gyros, accelerometers, tilt sensors and the like. The terrain compensation module is essential to navigate inside the field, but not always the GNSS-based guidance systems provide full autonomy to the vehicle. An intermediate solution, known as semi-autonomy situation, provides assistance to the operator who is actually driving the vehicle but following the indications of the automatic navigator [11, 16-18]. This is a fairly common solution, mainly deployed in the first commercial units, and is typically realized through the lightbar displays. The lateral displacements of the vehicle are indicated by lighting an equivalent number of lights on the proper half of a LED bar. Lightbars are often replaced by more sophisticated monitors where a Graphic User Interface (GUI) provides the driver with the important data for a safe operation. The great accuracy demands of precision agriculture applications have resulted in the development of high performance GPS solutions capable of localizing a receiver with sub-inch exactitude. The method described in [18] integrates a Real- Time-Kinematics (RTK) GPS with precision farming to provide accurate seeding, cultivating, planting or harvesting. Due to its accuracy, this system can make use or assist in the construction of Geographic Information Systems (GIS) with 172 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms the overall goal of generating a digital map including the precise position required by each operation. 4. OPTICAL AND LOCAL PERCEPTION After the initial euphoria brought by the easy and reason- ably priced availability of GPS data, significant challenges with potential severe consequences became visible and resulted in the necessity of local perception for vehicles operating with some degree of autonomy. Bias, drift and guess-row errors led Rovira-Ms et al. [19] to postulate a model for sensor fusion of machine vision and GPS to guide a tractor at various speeds and under diverse field and sensor signal quality conditions. Optical range finders are universal solutions for estimating distances as a way to perceive locally, that is, in the vicinity of the vehicle. These sensors emit electro- magnetic radiation and receive a reflecting beam from the detectable object; a timer determines the elapsed time that allows the calculation of distances. The electromagnetic radiation can be infrared (IR), near infrared (NIR), ultra- violet (UV), red light, the visual spectrum, etc. In a general embodiment [20, 21], the outbound optical path consists of the electromagnetic source, the focusing optical member (lens), and a micro-mirror array and transmission optical member. The inbound optical path includes the reception optical member and a detector. A processor is in charge of converting elapsed time into distance, and to do so it may consider the first return, the last return, the strongest return, the average of returns, or the median of returns. Sensing one object can be helpful for obstacle avoidance, but guiding a vehicle requires multiple distance measurements. The application patented in [22], for example, utilizes a plurality of markers located on a predetermined road; a method coined continuous/discrete marker system. The markers are reflective stripes painted along the path. The separation bet- ween reflectors depends on the design and vehicle speed, but the inventors suggest in one of the prototypes described a spacing of 50 feet. The signal processing algorithm, calcu- lates the heading angle with respect to the approximated path as a mean to determine the desired steering angle of the vehicle. The previous descriptions set the common frame- work for local sensing, but the specific applications can be divided into two groups: computer vision and the remainder of range finders. 4.1. Computer Vision The tremendous potential of machine vision as a perception-navigation sensor for off-road vehicles, due in part to its similitude to human visual perception, has resulted in the coalescence of agricultural and non-agricultural institutions pursuing similar goals regarding vehicle auto- mation. This is the case of the self-propelled harvester robotized by Carnegie Mellon University [23, 24]. The harvester was endowed with a crop-tracking apparatus for steering the vehicle along the crop without operator assis- tance, although, for safety principles, the presence of the operator was required in the cabin when navigating under automatic mode. A video camera was set on a protruding arm at 4 m height to detect the cut-uncut edge (crop line) of the yield being harvested. A color-based discriminator func- tion running on a microprocessor found the best position for the crop line. The harvester executed the turns by powering the front wheels at different speeds with hydraulic motors. The kind of scene captured by the vision sensor shapes and defines the image processing algorithms from which decisions will be made. In the case just explained, the searched feature was the crop line delimited by the cut-uncut front, one of the most common scenarios found in agricul- tural fields. Another familiar in-field situation is the important case of crops arranged in rows. For this condition, the system proposed in [25, 26] features four regions of interest (ROI) to identify two crop rows robustly and guide a tractor. A region of interest is a window selected within the image where image processing techniques are applied to extract meaning- ful information, in our particular case about navigation. The series of images depicted in Fig. (2) illustrates a typical image analysis for computer vision navigation of a tractor in a soybean field [27]. The distinction between crop and non- crop (mainly soil) implemented in [25, 26] was based on the k-means algorithm applied to the four regions of interest. The k-means algorithm is a clustering technique which minimizes the within group variance to partition the cluster, where k represents the number of groups or classes. Once the partitions have been realized, the algorithm applies moment operations to determine the location and orientation of crop rows. Linear regression is then used to define the trajectory path. An alternative processing technique to identify the crops and find the optimum path is based on the Hough transform [27], applied to the image shown in the example of Fig. (2). Machine vision techniques have been combined with other algorithms more representative of artificial intelligence, but really effective to cope with the usual difficulties for an imaging device operating in open fields. A two-class k-means algorithm fed a fuzzy logic controller to decide the best estimate for the slope and intercept of a line representing the cut-uncut edge detected by a video camera mounted on a harvester [28]. Due to the fact that the camera created a perspective shift growing with the width of the combine head, and contrary to the case reported in [23, 24], the camera was eventually mounted on the combine head, making the edge an almost vertical line. In the headlands, though, there are no cut-uncut boundaries and the farmer needs to steer the vehicle in a semicircular path until it faces the boundary from the opposite direction. Since direction is reversed, the camera is too far to capture an adequate view, and therefore a second camera was installed in the opposite side of the corn head so that the vehicle could harvest from both sides [29]. A third situation, in addition to the detection of har- vesting cut-uncut lines and crop rows, is represented by the automation of a self-propelled picker following a cut swath of crop lying on the ground [30]. Given that the only source of guidance features is the swath, the ROI needs to be precisely defined to avoid the swath falling outside the processing window. A successful technique to deal with the swath detection challenges classified swath pixels according to their texture information in relation to the texture of the surroundings; the vertical stalks of grass remain standing while the swath includes longer stalks in multiple directions, hence the difference in texture. A grayscale dependency Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 173 matrix based on 8-connectivity was the preferred operation to find the swath in the input images. When the color discri- minator of the harvester described in [23] found problems with the Sudan grass, a texture discriminator based on the Gabor filter was implemented instead. The localization information deduced from image analysis can be compli- mented with other navigation parameters such as the odometry-based traction control on farm vehicles disclosed in [31]. Odometry measures travel over a surface and is normally used in dead-reckoning navigation. The accuracy of this distance estimation may be improved with the assistance of GPS position information. The participation of GPS technology enhances the reliability of the measurements because the optical odometer is a differential instrument and accumulates errors over distance. 4.2. Other Range Finders One of the classic alternatives to computer vision for vehicle local perception is represented by lasers range finders. The laser radar for lateral guidance reported in [32], produced a pulse modulated beam and received reflected light beams from mounted retroreflectors, covering the portion of the road ahead of the vehicle. The distance and angle to each reflector were calculated to determine the position of the vehicle and to estimate the road curvature for commanding course correction information. The road references may be standard roadway reflectors mounted at regular intervals of, say, 15 meters apart. A conventional laser transmitter may be a medium power IR (100 W) centered at a wavelength of 820-830 nm. The other broad-spectrum local sensors for mobile robotics are ultrasonic range finders. An ultrasonic guidance apparatus envisioned by Coker [33] is capable of steering a vehicle on a desired track between adjacent crop rows or sides of a marked furrow. Onboard sensors are setup for transmitting ultrasonic ranging signals and receiving the reflected signals from the rows or furrows. Two ultrasonic emitters need to point downwardly and outwardly in opposite directions towards adjacent crop rows or plant stalk bottoms; as the vehicle moves laterally, the distance from one sensor will increase while the distance to the other will decrease. A transducer measures the elapsed time of the ultrasonic pulse to travel to the object and back to the sensor, computing the distance based on the speed of sound. If both transducers sense an increase of distance, the data is disregarded because it indicates a vertical motion (bounce) rather than a lateral displacement. This approach is, to a certain extent, reactive and in some way reminds of mecha- nical feelers. 5. GUIDANCE ALGORITHMS, PATH-PLANNING, AND VEHICLE CONTROLS The birth and development of the concepts key to artificial intelligence has preceded the availability of fast, efficient, and cost-effective robotic hardware, yet it has been the fast growth of electronics and sensors what boosted automation in off-road environments. Whatever the case may be, any vehicle bestowed with some degree of autonomy requires (artificial) intelligence, and the design of guidance algorithms and control strategy is probably the central stage in automatic navigation. This section reviews the most used approaches to process sensor data and elaborate an actuation plan to drive the vehicle as safe and as accurate as possible. Different sensing techniques might make use of the same control stratagem to steer a vehicle and vice versa; the same localization and perception data can be interpreted in many different ways to guide an autonomous vehicle. Being computer vision one of the preferred techniques for vehicle local perception, there is a rich body of vision-based algo- rithms to resolve a vehicles trajectory. The self-propelled harvester described in [23] couples a best fitting step function to the harvesting frontline using the least-squared error criterion and a vote counter. The position of the crop line directly establishes the turning angle of the wheels, which is executed by rotating the front wheels at different rate. The multiple ROI guidance system for rows tracking covered in [25, 26, 34-36] is able to trace a trajectory expressed in a matrix with a discrete number of points, the first five being extracted from the input image. A fixed look- ahead time is employed to estimate look-ahead distances. The trajectory path is determined via linear regression, and the planner eventually finds the desired wheel angle and the look-ahead distance along the y-axis (forward direction). The same k-means algorithm but with k =2 [28] was used to track the cut-uncut edge of corn being harvested. The separation line was determined by regression analysis and the acceptability of the solution evaluated through fuzzy logic membership functions: a perfect vertical line (infinite slope) indicates that the camera is located over the cut-uncut edge [29]. A PID steering controller governs the hydraulic cylinders of the steering mechanism in the rear wheels of the combine. Commands from the controller excite a solenoid that actuates a proportional control valve.
(a) (b) (c) (d) Fig. (2). Typical image analysis for navigation of off-road vehicles: (a) Region of interest selection; (b) Segmentation; (c) Mid-point encoding; and (d) Row identification and optimal path [27]. 174 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms Vehicle navigation entails the definition of the desired destination to be reached by the automatic machine, and this can range from a simplistic reactive system such as mechanical feelers to a sophisticated trajectory planner. An example of an intermediate case is represented by the lateral guidance control system described in [37], where the calculation of the target point is based on the length of the preview distance obtained as the intersection of the radius of half circle with road geometry. Steering angles are computed with linear equations as functions of marker gains and heading angles. The multisensory harvester equipped with GPS, INS, and two video cameras of [38] followed a field coverage plan generated by a trajectory planner. The targeted speeds are about 4 or 5 miles per hour (6-8 km/h) and the main controller follows a behavior-based approach. This control strategy was conceived by Rodney Brooks [39] and establishes a direct link between perception and actuation, circumventing cognition by high-level reasoning entities. Conventional wisdom establishes that before reaching the state of full autonomy, there will be many ideas, applications, prototypes, and solutions regarding semi- autonomy. The majority of the works reported so far request the presence of drivers at the extremes of the fields to perform turns between passes, and a good policy for engaging and disengaging automatic control becomes a matter of efficiency. The method developed in [40] initializes automatic guidance after turning at the end of the path. When the operator moves the steering mechanism, autoguidance deactivates to allow the driver perform the turning maneuver; after it, the position of the steering mechanism is monitored, and if it falls within a predetermined zone, autoguidance remains activated. Other- wise, it is disconnected. This system lets automatic control remain connected when the machine is turning and the steering mechanism is not in the straight travel position, regarded as the deadband zone. The deadband zone encom- passes several degrees on either side to account for small variations in steering. Once automatic control is engaged, most of the systems rely on fluid power to control the vehicle. The harsh and force-demanding nature of off-road equipment duties places hydraulics as the preferred actuation mode. The vehicle mentioned in [14] features a PID controller to move from one waypoint to the next, getting feedback from a wheel angle sensor. The robotized tractor of [41] possesses various sensors whose data is fused for real- time guidance. It combines map-based guidance and sensor- based guidance because no individual sensing technology is ideally suited for vehicle automation. Data fusion is carried out with the Kalman filter. The actuator is, again, an electro- hydraulic valve commanded with Pulse Width Modulation (PWM) signals. The wheel angle sensor is a rotary potentiometer aligned with the king pin on the right wheel. The basic difference between semi-autonomous and auto- nomous guidance rests on actuation; the rest of the process remains the same. Therefore, both systems may include similar localization and perception systems, identical control algorithms and trajectory planners, but only the latter is expected to implement actuating mechanisms to guide the vehicle, typically through hydraulic cylinders. A popular semi-automatic solution is represented by the so-called assistive systems. The embodiment of one of such systems [42, 43] utilizes a feedback control system to compare the actual position of the vehicle with the desired path, providing feedback to the human operator through a visual display. The enhancement in efficiency conducting regular farming tasks is considerable with assistive systems. The complexity of the control algorithm is comparable to those implemented in autonomous navigators, and the assistive system mentioned above, for instance, proposes a third order scheme (three poles) for under-damped response with variable (velocity dependent) gains k p , k s , and k h . Very often, the output data from localization or percep- tion engines directly feed the vehicle controller to execute the essential functions of steering, braking, or accelerating. However, some situations and complex solutions insert a trajectory planner in-between. One non-trivial scenario is found when a vehicle is guided along adjacent swaths across terrain with slope [44]. The solution can be outlined in three consecutive steps: first, estimate the tilt of the vehicle due to the terrain slope; second, use GPS to find the position of a fixed point on earth; and third, from tilt and position measurements, calculate guiding commands to prevent gaps and overlaps. A complete solution can be understood as the one comprising curved and straight segments, headland turns, and a plurality of modes. The vehicle foreseen in [45] accepts three operation modes (manual, semi-autonomous, and autonomous), projects a plurality of trajectories (parallel tracking or several row skipping patterns), and adapts the forward velocity to the position of the vehicle in the field (turn-entry velocity, turn velocity, and turn-exit velocity). The region-filling module (path planner) weaves the path pattern within the perimeter defined by the user. The A *
algorithm is the internal engine in charge of producing the interconnected segments that form the preferred path [46]. In the embodiment of this complete solution, the path planner actually comprises a perimeter training module which collects location data and a contour region-filling module [47]. The starting input data for the planner is the vehicle minimum turning radius, the task width (note that vehicular width may vary from implement width), and the reference row location. The interference module establishes geometric constraints between adjacent arcs to avoid interferences, that is to say, when two contiguous components converge and there is no room to fit both. The reference contour must be chosen in such a way that the energy consumption for completion of coverage is minimized. The reference path is defined by the coordinates and heading of the starting and final points; once known, the planner identifies candidate paths. An estimator calculates an economic cost for traver- sing each candidate path. The economic cost depends on distance and duration. The estimator may eliminate a path if it is not drivable, collides with obstacles, or the economic cost is above a determined threshold. By applying these rules, the estimator is capable of eliminating unnecessary looping [48]. For certain regions, spiral paths may be more efficient than linear or other paths in terms of time, fuel, or the like. In order to cope with these situations, a spiral region-filling module is designed [49] with the potential to map out contour rows following a spiral, plan a path comprising nested loops, or generate a curve of variable radius. Spiral shapes might have problems with some work areas, such as those with pinch spots, and require a partial Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 175 application. If the work area is mostly circular, an Archi- medean spiral may be used. When contours run adjacent to a long side of the region, the spiral tends to minimize flip turns to reduce time. When the desired path is defined by a set of waypoints, and the turning radius of the vehicle can imperil the completion of the mission, sophisticated courses need to be taken such as the bi-variate periodic bi-cubic -spline fitting curve, which is known to yield a smooth curve between waypoints [50]. With complex geometry, such as spirals and loops, it might be confusing to separate the regular tracking course from headland turns. The method developed in [16] specifically defines turns as changes in heading 30 over a distance of 100 m. 6. PRECISION AGRICULTURE AND IMPLEMENT CONTROL It seems somewhat contradictory to increase accuracy in a vehicles localization system, by means of RTK-GPS for example, while incurring in gross errors controlling the implements as a consequence of key sensors being mounted on the vehicle and not on the tool executing the task. This problem becomes more severe as machinery grows. Incidentally, outsized terrains are the most benefitted from precision farming applications, but on the other hand, they usually require large-scale equipment. For this reason, implement control is becoming a recurrent topic in this novel farming technique. Precision agriculture is, from and by its genesis, inseparable from global satellite localization. Furthermore, very frequently it combines real-time GNSS information with GIS and aerial photographs of farms to create rich digital maps. A generic example of a site-specific farming system is given in [51], where the boundaries of agricultural fields are identified to execute functions depen- ding on the position of the vehicle: variable-rate application and implement height depends on boundary information. The speed of the vehicle can be adjusted according to the distance of the vehicle to the boundaries. Implement ele- vation, as it will be in most of the applications described below, is controlled with relays, solenoids, and PWM gene- rators. Information technology (IT) has enriched the corpus of data handled by intelligent machines, and consequently autoguidance is, far from an isolated and curious feature as in the beginning, another characteristic of a site model of the field, where a path planning system is adapted to the model, the implement, and the operation [52]. The site model manages parameters such as yield, rainfall, prescriptions (rate, implement elevation, and application speed), boundary information, and obstacles data. During normal operation, the vehicles also stores data for the site model of the field, which is allocated in the site database, the operations database, or the implement database. Nevertheless, precise or site-specific applications can manage without satellite positioning in especially controlled atmospheres. Let us con- sider the case of a self-navigating robotic vehicle enclosed in a garden with undulated irregular terrain [53]. The vehicle can follow a specified course by dead-reckoning if multiple navigation beacons are placed all around the perimeter of the garden. The navigating module can estimate position and orientation with accuracies of one inch by just processing the images captured with a camera detecting the beams from the beacons. The robot controls work implements without human intervention according to a pre-stored scenario called the work order. The work order can include route, speed, turning radii, implement control points, etc. This mode of establishing a tasking protocol for the vehicle applies industrialization approaches to agricultural production. The invention proposed by Hanley [54] applies flexible manufacturing software, robotics, and sensing techniques to agricultural operations. The idea is to move programmable workstations typically used in manufacturing operations to farm fields through mobile robots that can sense their environment and respond with the typical flexibility of manufacturing processes. The specific tasks considered are pruning, harvesting, weeding, trimming, planting, spraying, pollinating, or any operation on plants comprising a mobile robotic machine with an arm and a decision making unit based on artificial intelligence. In this approach, a human operator is essential either locally or remotely, with some autonomy also available. Captivatingly enough, it also points at the convenience of this application to specialty crops, a production niche that every day is demanding more robotic solutions to lower labor costs while augmenting selectivity and adaptability, or in other words, intelligence. Aerial photographs and GIS-generated maps cannot catch up with the dynamism involved in the active production of farms run by precision agriculture principles. The ideal situation is mapping while working in such a way that the set of spatial variability maps are always being updated in real time. This is the case of [55], where a supporting member with a plurality of sensors is set to gather perception information in combination with the vehicles position to create a map of the field, as the presence or absence of weeds at specific locations to build a weed-map. The mapping unit is connected to the control unit and a display via CAN protocol, so that speed and optical sensors are able to open and close spray nozzles at the right time. In addition to regulating chemicals or irrigation water, height control in implements is crucial for automating farming operations. The control circuit designed in [56] generates signals to raise and lower agricultural tools such as plows and combine heads. A height sensor is coupled to the control circuit to sense the implements height and therefore provide the feedback signals required by a closed loop control. This automatic control decreases operator workload while increa- sing accuracy in comparison to manual control. Obviously, the higher precision in controlling the implement, the higher accuracy in the operation, therefore it makes sense to invest in improving sensing and actuating technology as a means to enhance productivity. One way of reinforcing localization quality is by coupling two positioning receivers with a power-actuated hitch to maintain an implement on course [57]. The hydraulic cylinder of the hitch is equipped with a potentiometer for position feedback. Another way to melio- rate the performance of automated tasks is by upgrading the positioning technology. The system reported in [58] integrates RTK-GPS with precision farming to provide accurate seeding, cultivating, planting, or harvesting. A seeder can be configured as a boom supporting one or more GPS antennas. If the antennas are placed at either end of the boom, the elevation of each end can be estimated. Precise seeding with RTK-GPS may provide means for micro-crop 176 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms rotation: rotation of the position of crop plants within a small area of the field, considering cm-level accuracy. The ultimate step in reaching high accuracy with the implement actuation is by mounting the GPS antenna (RTK if feasible) directly on the farm implement, and coupling the antenna to the receiver [59, 60]. The implement has the tendency to displace to the side opposite the direction that the tractor is steered. This implement guidance system provides a fast response for dynamically adjusting side-to-side positioning of the implement. The automatic hitch includes an actuator and a position sensor. The actuators are dual hydraulic devices for pushing and pulling the tongue of the hitch. The control computer reads the geographic information of the path from memory and determines the cross track error of the implement rather than the vehicle. The authors claim that this farming system corrects the location of the implement in less than one half the time or distance required with ordinary on-vehicle guidance. The computer handles two parallel operations: vehicle guidance and implement guidance. An alternative, non-GPS related, attention-grabbing control of a guided implement with extending arms for spraying the ground is illustrated in [61]. This technique uses fluorescent chemicals to mark regions, to be exact 4-6 inches wide bands, and determine the position of the implement to keep it on course. The objective is to avoid overlapping and underlapping of chemicals. The spraying boom requires a nozzle for depositing a band of fluorescent chemical, which in turns will be excited by a light source. A hydraulic motor governed by a proportional valve brings the implement on track. This system can control large implements with booms over 120 feet. 7. SYSTEM RELIABILITY AND SAFEGUARDING Safeguarding is probably the factor that has delayed the desired transformation from concept to product the most. Rural environments, where heavy and powerful machines regularly operate, are potential candidates for mortal accidents. Leading manufacturers do not want to have their reputation damaged and consequently tend to be conser- vative in the implementation and commercialization of new technology, mainly with regards to vehicle automation. Errors accepted to drivers are not accepted to machines which usually need to outperform skilled operators. The wide scope of onboard safety systems ranges from basic semi-autonomous applications to sophisticated robotic solutions. The invention covered in [62] is designed to aid humans in the safe use of equipment by deploying a messag- ing network and monitoring remote control operations. The system includes a sensory device for monitoring and reporting data, a memory device, at least one processor, and a plurality of external devices such as pagers and cell phones to communicate via security protocol with encryption technology. A probable hazard for an automatically guided vehicle performing a task in the field is the presence of an unexpected obstacle interfering with its trajectory. The natural reaction for the vehicle would be to detect the obstacle and move around it if there is enough space. In order to do so, the controller must contain an obstacle avoidance algorithm. The point-to-point path planning of [48] is capable of detecting obstacles and defining a boundary zone about each obstacle. The obstacle clearance zone is defined by modeling a perimeter as a percentage- augmented periphery and is based on such vehicle cons- traints as width, wheelbase, and turning radius. The planner identifies candidate paths and eliminates those colliding with obstacles. The number of touches of the obstacle zones is limited. Another method for getting information about objects in an environment around a vehicle uses infrared (IR) light to create a 3D representation of the surroundings based on the measured distances to the objects [63]. The security system can be amplified with the addition of an ultrasonic system in an optical-acoustical combination. Dangerous situations are warned by turning on the hazard lights, soun- ding the horn, or similar actions. A remarkable advantage of this system is that even in the presence of strong sunlight, a narrow beam of IR laser can be observed if using a narrow notch frequency filter on the received light. This embo- diment is recommended for air seeding systems, precision planters, and sprayers. When the vehicle approximates too much to an interfering obstacle, stronger warning systems need to be activated. The anticipatory sensor system of [64] informs that an impact involving the vehicle is about to occur unless a corrective action is taken. This action is based on data obtained prior to the impact. Potential applications of the system include protection to pedestrians, lane departure warning signaling, and night driving assistance. Lasers, cameras, GPS, and IMU are some of the sensors employed to presage an impact. In its adaptation to agricultural vehicles [65], a ground speed sensor is mounted onboard to calculate wheel slip of the tractor or sprayer. The obstacle detection algorithm utilizes pattern recognition techniques to identify the dangerous objects [66], and some of the habitual processing methods are neural networks, Fourier analysis, or fuzzy logic. In addition to reactive trajectory re-planning induced by local perception sensors, when global positioning is part of the navigation system, reliability can also be improved by tracking a diagnostic function to ensure that the navigation system is working properly [14]. This function can include the number of satellites, quality indices such as de dilution of precision, CPU load, or even the solenoid response. 8. CURRENT & FUTURE DEVELOPMENTS In spite of more than two decades of intense and fertile development of automatic guidance technology, we can consider this technology a young technology from the customer acceptance standpoint; if it is true that some early adopters are regularly using some of the few commercial systems available in the market, the majority of professional producers still remain stuck to conventional systems and procedures. There is still a long way to go, although users grow year after year and the trend seems to be favorable. The incorporation of new satellite systems to the arena of GNSS, such as GLONASS and Galileo, will probably lead to a wider use, perhaps universal, in agriculture, forestry, and construction. The side effect will certainly be the increase in complexity of receivers which will need to accept different sources of signals, but on the other hand, reliability will also be enhanced as redundant signals result in safer outcomes. This GNSS augmentation will have direct repercussions on precision agriculture future paths as spatial variability will be better tracked with the new localization data abundance. Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 177 Western countries seem to find troubles with conventional agriculture while developing countries need technology for a sustainable production; precision farming appears as a factual solution to assist both needs. The incorporation of these technological advances, however, will require several steps. Hardware and software reliability needs to be guaranteed for the harsh environments found in agricultural fields. Extension personnel should smooth the transfer from research laboratories to farming sites. Manufacturers, in conjunction with dealerships, have to be prepared to instruct and assist their customers in the operating techniques and data handling. The analysis of the technological evolution around intelligent vehicles can bring some insights to predict what might occur in the next decade. The bar chart of Fig. (3) represents the weight of key features found in automatic guidance patents according to their period of publication: 1985-2000 and 2001-2008. It is apparent in the graphic that beacons, pseudolites, and non-vision local sensors have experienced a recession in favor of computer vision systems and algorithms, probably motivated by the significant growth in computer power. Inventions related to precision agri- culture were more numerous in the first period, which is surprising because selective availability disappeared after 2000. It may be possible, though, that the loss in GPS signals restrictions has weakened research devoted to compensate for signal degradation. Nevertheless, the applications of GPS systems, as expected, have increased significantly, with 80% of the patents citing this technology being published after 2000, and 90% of the inventions dealing with parallel tracking also belonging to the period 2001-2008. It is note- worthy the fact that most of the semi-autonomous appli- cations reported in the reviewed patents have been developed in the second period, and lightbars guiding assistance is dominant in the second period as well. After the oneiric start of vehicle automation where some people envisaged farms of crowded machines multitasking on their own, roboticists and engineers found a modest commencement based on the deployment of semi-autonomous equipment a more reason- able approach for consolidating this technology. Guidance visual assistance through lightbar displays and straight-alone autopilots turned out to be grounded solutions to begin with. Headland turns and field-filling complete trajectories are in the agenda but still need the support of highly safe software and hardware. In light of these evidences, the coming years will probably see a continuation of GNSS applications with better signals and filters, a perfection of imaging sensors, and the expected enhancement of reliability and safety. Sensor fusion of global and local information might also step up. The Grand Challenge competition demonstrated that the possibility of endowing a vehicle with advanced intelligence is realistic; the required technology is available and reason- ably reachable. Yet the competition was a giant laboratory where high-tech prototypes were defied, and it will take some time for manufacturers to incorporate automatic gui- dance kits in their mass-produced models. Some of the initial problems such as sensor accuracy, processing speed, or cost are being fixed with time; but reliability is still an unsolved
Fig. (3). Technological evolution of auto-guidance for the periods 1985-2000 and 2001-2008. 178 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-Ms problem. For this reason, perceptual devices will expand their horizons, and more sophisticated sensors as stereo- scopic vision heads or 3D laser scanners, perhaps borrowed from space technology, will join the already numerous family of hardware integrated in intelligent vehicles. In any case, the essence of a vehicles behavior, its intelligence after all, will be dictated by the onboard algorithms and program- med conducts, and from that stand-point, significant advan- ces are expected in the philosophical conception of the vehicles brain. ACKNOWLEDGEMENTS I would like to express my most earnest gratitude to J ohn Reid, Qin Zhang and Shufeng Han for being the individuals who introduced me into this fascinating field, taught me the essentials, and motivated me to continue within its boun- daries and even transcend them. CONFLICT OF INTEREST None known. REFERENCES [1] Reid J F, Zhang Q, Noguchi N, Dickson M. Agricultural automatic guidance research in North America. Comput Electron Agr 2000; 25: 155-167. [2] Shirley R. Ventura Countys Changing Agriculture. Resource; February 2008: 5-8. [3] Gibbs WW. Innovations from a robot rally. Sci Am 2006; 294(1): 64-71. [4] Pryor, T.R.: US4769700 (1988). [5] Dyke, H.L.: US4700301 (1987). [6] Barnes, R.L., Gary, D.A.: US5121799 (1992). [7] Rocks, J .K.: US5974348 (1999). [8] Hale, G.H., Wendte, K.W.: US5870689 (1999). [9] Misra P, Enge P. Global Positioning System. Signals, Measure- ments, and Performance. Ganga-J amuna Press: Lincoln, Mass, USA 2006. [10] Rovira-Ms F, Han S, Reid J F. Trajectory-based algorithm for the quantitative evaluation of automatically steered vehicles. Trans ASABE 2008; 51(5): 1545-1552. [11] Goodwin, J .L., Hartt, J .R., Vetter, A.A., Walton, J .T., Bletzacker, F.R., Forgette, T.M., Hughes, A., Neuperger, F.H., Falkenberg, W.H.: WO9515499 (1995). [12] Mullins, D. B.: US6377881 (2002). [13] Parkinson, B.W., O'Connor, M.L., Elkaim, G.H., Bell, T.: US20006052647 (2000). [14] Mailer, R.L.: US6876920 (2005). [15] Nichols, M.E.: US6501422 (2002). [16] Lindores, R.J .: US7363154 (2008). [17] Lindores, R.J .: US20070083299 (2007). [18] Keller, R.J ., Nichols, M.E., Lange, A.F.: US6199000 (2001). [19] Rovira-Ms F, Han S, Wei J , Reid J F. Fuzzy logic model for sensor fusion of machine vision and GPS in autonomous navigation. ASABE Annual Meeting 2005: Paper No. 051156. [20] Anderson, N.W., Thackray, R.G.: US7064810 (2006). [21] Anderson, N.W., Thackray, R.G.: US7206063 (2007). [22] Margolis, D.L., Yasui, Y.: US5357432 (1994). [23] Stentz, A., Hoffman, R., Ollis, M., Whittaker, W., Fitzpatrick, K.: EP0801885 (2002). [24] Stentz, A., Ollis, M., Fitzpatrick, K., Hoffman, R., Whittaker, W.: US5911669 (1999). [25] Dickson, M.A., Ni, B., Han, S.: US6285930 (2001). [26] Dickson, M.A., Ni, B., Han, S., Reid, J .F.: US6385515 (2002). [27] Rovira-Ms F, Zhang Q, Reid J F, Will J D. Hough-transform-based vision algorithm for crop row detection of an automated agricultural vehicle. J Automobile Eng 2005; 219: 999-1010. [28] Benson, E.R., Zhang, Q., Reid, J .F.: US6721453 (2004). [29] Benson, E.R., Zhang, Q., Reid, J .F.: US6819780 (2004). [30] Hofer, J ., Bohner, M., Freichel, T.: US7400957 (2008). [31] Weinstein, L., Sinclair, K., Gainsboro, J ., Willison, P.: WO2005084155 (2005). [32] Frank, A.A., Nakamura, M.: US5202742 (1993). [33] Coker, W.B.: US5410479 (1995). [34] Dickson, M.A., Ni, B., Han, S.: US6686951 (2004). [35] Dickson, M.A., Ni, B., Han, S.: US6278918 (2001). [36] Dickson, M.A., Ni, B., Han, S.: US6490539 (2002). [37] Margolis, D., Yasui, Y.: US5390118 (1995). [38] Pangels, H., Pilarski, T., Fitzpatrick, K., Happold, M., Ollis, M., Whittaker, W., Stentz, A.: US6336051 (2002). [39] Brooks RA. Cambrian Intelligence. The Early History of the New AI. The MIT Press: Cambridge 1999. [40] Motz, D.S., Saxsma, A.L., Staub, M.D.: US6738695 (2004). [41] Dickson, M.A., Noguchi, N., Zhang, Q., Reid, J .F., Will, J .D.: US6445983 (2002). [42] Harrison, J r. J .H., Harrison, H.J .: US6925380 (2005). [43] Harrison, J r. J .H., Harrison, H.J .: US20060089796 (2006). [44] Nichols, M.E.: US6703973 (2004). [45] Gray, S.A., Hansen, S.L., Flann, N.S.: US6907336 (2005). [46] Gray, S.A., Hansen, S.L., Flann, N.S.: US7110881 (2006). [47] Flann, N.S., Hansen, S.L., Gray, S.A.: US7216033 (2007). [48] Flann, N.S., Hansen, S.L., Gray, S.A.: US7079943 (2006). [49] Flann, N.S., Hansen, S.L., Gray, S.A.: US7228214 (2007). [50] Overschie, J .: US7277792 (2007). [51] Hale, G.H., Wendte, K.W.: US5978723 (1999). [52] Motz, D.S., Staub, M.D.: US6236924 (2001). [53] Rocks, J .K.: US5974348 (1999). [54] Hanley, B.P.: US6671582 (2003). [55] Beck, J .L., Kinter, M.L.: US5809440 (1998). [56] Hale, G.H., Wendte, K.W., Orbach, A.: US5961573 (1999). [57] Mcclure, J .A., Heiniger, R.W., Timm, J .T.E., Funk, K.D., Wong, R.B.: US7373231 (2008). [58] Keller, R.J ., Nichols, M.E., Lange, A.F.: US6199000 (2001). [59] Lange, A.F., Funk, T.J .: US7054731 (2006). [60] Lange, A.F., Funk, T.J .: US7383114 (2008). [61] Sauter, G.F., Nelson, G.F., Andersen, D.P., Paulinski, D.W.: US5636792 (1997). [62] Walker, R.C.: WO01054044 (2001). [63] Breed, D.S., DuVall, W.E., J ohnson, W.C.: US7209221 (2007). [64] Breed, D.S.: US20080040004 (2008). [65] Breed, D.S.: US20050278098 (2005). [66] Breed, D.S., DuVall, W.E., J ohnson, W.C.: US20070182528 (2007).