Warfighter Research in Mine Detection: Improved Clutter Rejection with the An/PSS-14 Todd R. Higgins, Ph.D.

Program Director/Principle Investigator James Staszewski, Herman Herman, Virgil Flanigan, Olga Falmier, and Robert A. Hancock Co-PIs Land Mine Detection Research Center Department of Computer Science, Technology, and Mathematics College of Behavioral and technological Sciences Lincoln University Jefferson City, MO 65101

LU/LMDRC

October 2008

NOTICES Disclaimers The findings in this report are not to be construed as an official Lincoln University or Department of the Army position unless so designated by other authorized documents. Citation of manufacturers or trade names does not constitute an official endorsement or approval of the use thereof. DESTRUCTION NOTICEDestroy this report when it is no longer needed. Do not return it to the originator.

Land Mine Detection Research Center

Department of Computer Science, Technology, and Mathematics, Lincoln University, Jefferson City, MO 65102

LU/LMDRC

October 2008

Warfighter Research in Mine Detection: Improved Clutter Rejection with the AN/PSS-14
Todd R. Higgins, Ph.D. Program Director/Principle Investigator James Staszewski, Herman Herman, Virgil Flanigan, Olga Falmier, and Robert A. Hancock Co-PIs Land Mine Detection Research Center Department of Computer Science, Technology, and Mathematics College of Behavioral and Technological Sciences Lincoln University

Approved for public release; distribution is unlimited.

REPORT DOCUMENTATION PAGE

Form Approved OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To)

10-13-2008
4. TITLE AND SUBTITLE

Final

July 2006 May 2008

5a. CONTRACT NUMBER

Warfighter Research in Mine Detection: Improved Clutter Rejection with the AN/PSS-14

5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER

Todd R. Higgins, Herman Herman, James Staszewski, Virgil Flanigan, Olga Falmier, and Robert A. Hancock

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Lincoln University 820 Chestnut Street Jefferson City, MO 65101
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

8. PERFORMING ORGANIZATION REPORT NUMBER

10. SPONSOR/MONITOR'S ACRONYM(S) 11. SPONSOR/MONITOR'S REPORT NUMBERS

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES

14. ABSTRACT

15. SUBJECT TERMS

AN/PSS-14, program of instruction, mine detection, clutter rejection

16. SECURITY CLASSIFICATION OF: UNCLASSIFIED a. REPORT b. ABSTRACT c. THIS PAGE 17. LIMITATION OF ABSTRACT 18. NUMBER OF PAGES 19a. NAME OF RESPONSIBLE PERSON

Todd R. Higgins
19b. TELEPHONE NUMBER (Include area code)

UNCLASSIFIED

UNCLASSIFIED

UNCLASSIFIED

SAR

82

573-681-6194
Standard Form 298 (Rev. 8/98)
Prescribed by ANSI Std. Z39.18

Contents
Contents List of Figures List of Tables Executive Summary Study 1. B. Method C. Results D. Discussion Study 2. A. Introduction B. Method C. Results References Study 3. A. Introduction B. Method C. Results D. Discussion 12 16 20 Error! Bookmark not defined. 12 16 20 Error! Bookmark not defined. 16 20 Error! Bookmark not defined. 5 7 8 9

Study 4. A. Introduction B. Method C. Results D. Discussion Study 5. A. Introduction B. Method C. Results D. Discussion Alphabetical List of Acronyms 12 16 20 Error! Bookmark not defined. 82 12 16 20 Error! Bookmark not defined.

List of Figures

1-1 1-2 1-3 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 2-10 2-11 3-1 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 5-1 5-2 5-3

AN/PSS-14 MD Investigation of a Target in Spiral Technique MD Investigation of a Low Metal Target With the Cross-Over Technique Position 0 Position 1 Position 2 Position 3 Position 4 Position 5 Detector Unit Harness GPR Response Curves for Target A GPR Response Curves for Target B GPR Response Curves for Target C GPR Response Curves for Target D Depiction of Robotic Mine Sweeper Testing Setup Consistent GPR Sim 25 (Oct 4) Inconsistent GPR Sim 12 (Feb 28) Target Placement Relative To the Center of the Detector Head Percentage of Passes with GPR Alert (21 Feb 07) Percentage of Passes with GPR Alert (15 Mar 07) (Burial Depth 1.27cm) Percentage of Passes with GPR Alert (15 Mar 07) (Burial Depth 5cm) GPR Auditory Alert Over Target, By Set, By Detector Comparison of the GPR Response of Two AN/PSS-14 Detectors Over the Same Sim 6 Target Comparison of the GPR Response of Two AN/PSS-14 Detectors Over the Same Sim 12 Target GPR Response Over Nail Placed Perpendicular to Detector Head Average Targets Found By Training Methodology and Gender Average Targets Located Within Allowable Distance By Training Methodology Average Distance of Chip Placement From actual Target Center

List of Tables
1-1 2-1 2-2 2-3 4-1 4-2 4-3 AN/PSS-14 Program Instruction Sweet Spot Offset Main Effects Model Interactions Model GPR Response To Clutter By Time, Date, and Clutter Orientation GPR Response By Detector Head Height and Target (Sim 6) offset (21 Feb 07) GPR Response By Detector Head Height and Target (Sim 6) offset (15 Mar 07) (Burial Depth 1.27cm) GPR Response By Detector Head Height and Target (Sim 6) offset (15 Mar 07) (Burial Depth 5cm) Differences Among Training Methodologies When N=27 Differences Among Training Methodologies When N=26 Time on Task By Training Method-Test Lane Only

4-4 5-1 5-2 5-3

Executive Summary
Purpose This report describes the findings of five studies conducted to optimize the clutter rejection performance of the AN/PSS-14. Introduction Fielding of the AN/PSS-14 dual sensor mine detector to the US Army and Marine Corps is ongoing. The AN/PSS-14s manufactures, Cy Terra, is working closely with the systems Program Manager to upgrade the systems software as issues are identified. In 2005, Lincoln University embarked on a research program focused on optimizing the performance of the AN/PSS-14 through research. The research has proceeded along two primary axis, training enhancement and robotic coupling. This report describes the findings of five independent research studies designed to assess training methods and robotic coupling. Findings In the first study, the duration of the program of instructions (POI) effect on clutter rejection training was assessed. The primary improvement the AN/PSS-14 brings to mine detection is its ground-penetrating radar (GPR). The GPR allows trained operators to distinguish between mines and non-mine (clutter) targets. In this study, false alarm rates (FAR) were looked at. Clutter rejection performance was compared between a five-day and four-day program of instruction. Clutter rejection significantly increased and FAR significantly decreased with the four-day POI. Study 2 focused on developing AN/PSS-14 operational expertise and training development support. This will be accomplished through the study of the kinematics of an AN/PSS-14 operators movements when conducting a GPR short sweep over a target. Technological limitations in a key measurement instrument limited the intended level of assessment, however a re-designed study illustrated that the patterns of GPR response were highly dependent upon individual targets. Study 3 was a task to construct a research robot capable of mounting and remotely operating an AN/PSS-14 during a robotic lane sweep. The robot was constructed and successfully trialed in navigating a mine lane using a Minelab F3 metal detector. Study 4 was designed to investigate the reliability of GPR alerts over mine stimulants at various detector head heights and target burial depths. The results proved to be extremely variable across a range of conditions. Lately, Study 5 investigated methods of skill sustainment training, comparing no practice and practice to a computer-aided instruction (CAI) system. Interestingly, as a group the research 9

participants receiving the CAI as their sustainment training performed the lowest on target detection when sweeping the test mine lane following sustainment training. A rationale for why the CAI group performed the least effectively is discussed. Conclusions/Recommendations GPR response to mine stimulants is variable. GPR variability appears to be influenced by a number of factors, although predictability of when GPR response is likely to be varied remains elusive. Recommend continued research into the factors that result in variability of GPR alerts. A four-day POI is effective in teaching AN/PSS-14 operator skills. Recommend a larger-scale study to validate research results. Computer-aided instruction was not an effective sustainment tool for the conditions used in this study. Recommend the computer-aided instruction research be conducted again under different study conditions -- more delay between training and testing. Recommend the computer-aided instruction set-up change from a table, chair, and mouse set-up to a virtual mine lane and a replica of an AN/PSS-14 as the mouse.

10

Study 1 Effect of Program of Instruction Length on Clutter Rejection

11

Introduction Background The AN/PSS-14 dual sensor mine detector was designed to produce a significant advance in countermine capability over the currently-fielded AN/PSS-12 metal detector (Santiago, Locke & Reidy, 2004). The AN/PSS-14 combines both a highly sensitive metal detector (MD) and ground penetrating radar (GPR) in a single unit that functions as a mine detector. In a comparsion with the AN/PSS-12, the AN/PSS-14 exhibited superior mine detection capabilities in environmental conditions that make the AN/PSS-12 ineffective, particularly environments with conductive soil. In addition, the AN/PSS-14 was designed to allow a trained operator to discriminate detector signals orginating from land mines (or land mine simulants) from detector signals originating from clutter, i.e, metal debris, as indicted by the slogan find the mine, not the clutter (Cyterra, 2006). Clutter rejection capabilities reduce the false alarm rate (FAR) thereby reducing mine clearance time. In a demonstration of the mine detection and clutter rejection capabilities of the AN/PSS-14, expert humaninatrian deminers had 100% probability of land mine detection and rejected 94% of clutter (United Nations Mine Action Service, 2006) in a Cambodian mine clearance operation.

Theory of Operation of AN/PSS-14 The AN/PSS-14 is a hand-held, stand-off mine detector that can detect both high metal and low metal mines (see Figure 1-1) in mineralized soils. The Sensor Head contains both a metal detector (MD) and ground penetrating radar (GPR). Signals produced by the MD and GPR are analyzed by the Electronics Unit (EU) which in turn produces auditory signals in the earpiece and an external speaker mounted in the EU. Auditory tones that can differ in loudness and pitch are generated by the MD in response to metal objects. A beep which is constant in pitch and loudness and readily distinguishable from MD tones is produced by the EU in response to GPR signals only when detectable metal is also present. Major components of the AN/PSS-14 are its battery case and cable (mounted on a belt worn by the operator), the electronics unit, earpiece and cable, control grip, wand assembly, and sensor head.

12

Figure 1-1 AN/PSS-14.

Detection of mines requires three skills or techniques completed sequentially: (1) lane sweep technique, (2) MD investigation technique, and (3) GPR investigation technique. These three skills allow the operator to complete three tasks: detection, location, and identification of targets in mine lanes. Lane sweep technique detects targets for investigation. MD technique locates the metallic center of detected targets. GPR technique properly executed over the center of a target allows the operator to identify the target as a mine (in training, a mine simulant) or clutter. Both the MD and GPR require training or calibration to the environmental conditions before successful operation. The calibration process requires a number of ordered steps (see Hancock, 2006, for a more complete description). GPR calibration, in part, is activated by a button on the Control Grip. Calibration is continuous until this button is released (see below). Sweep Technique Target Detection Proper sweep technique ensures that every square inch of a lane is correctly investigated for mines. The operators movement must allow him to cover the whole lane. Proper technique requires that: (1) the Sensor head be parallel to the ground as closely as possible head height should never exceed 2 inches; (2) the Sensor head must be moved across the lane in a straight line; (3) the Sensor Head must traverse the lane at a constant speed between 1.0 to 3.6 ft/sec; (4) the Sweep must extend laterally one-half of the sensor head width outside the lane; and (5) the Sweep must advance forward one-third of the sensor head width to begin the return sweep. Lane sweep is terminated with an alert, i.e., an auditory signal that a target is present. This alert can be either a variable tone output (frequency and volume of tone vary) from the MD unit or a GPR alert tone (standard tone with no frequency or volume variation).

13

MD Technique Target Location Once a target has been detected, a MD footprint must be developed. The MD footprint is developed so that the center of the target can be located. Following an MD alert, the operator pushed the detector head into the MD halo to get a clearer MD signal with which to make this frequency discrimination. Once the frequency discrimination was made, the operator returned to the point at which the MD alert was found and completed the appropriate MD technique. The MD techniques are described below. Upon location of the target center by either of the techniques described below, the research participants (RP) were trained to place a white chip at the target center. Following the completion of Target Identification or GPR Technique, this white chip was replaced by a chip of the color appropriate to the Target Identification. The MD footprint is the entire area in which a continuous MD signal is being generated. The spiral technique is summarized in figure 1-2.
Mine body (GPR signal) Halo (MD signal) MD Alerts

9 oclock approach

3 oclock approach

6 oclock approach Figure 1-2 MD investigation of a target with spiral technique. Used for all targets in 2005; used for high metal targets, only, in 2006. (Not drawn to scale.)

The operator must move the detector head in a semi-circular fashion around the target, establishing a small spiral pattern to find the outer boundaries of the signal or halo (see Figure 12) of the target or targets. This boundary can be established most easily by the onset (presence vs. absence) of the MD signal. The operator places white poker chips at the 3, 6, 9 oclock and other boundary positions of the target signal to help him visualize the target metal footprint. Once the footprint is built, either by chips or through a mental image, the operator lays a single chip (white for clutter, red for a mine, and orange for inconclusive) to declare the center of the target based on its metal signature. Please note that the high MD signal is the product of the mine simulants and mines with large amounts of metal, which is usually ferrous, in the composition of the mine or mine simulant. In addition, because most clutter is ferrous, clutter also produces a high MD signal. In the study conducted here, the material composition of all clutter was known and was ferrous. The low 14

MD signal is the product of mine simulants and mines with very small amounts of metal, primarily an aluminum tube. [There are two different metal components for the mine simulants employed by the U.S. Army and this study. The K insert, which consists of a 0.100g steel pin inside of a small aluminum tube (0.50 inch length x inch O.D x 0.015 inch wall thickness), is used in anti-tank low metal mine (ATLM) simulants. The I insert, which consists of an aluminum tube (0.5 inch length x 0.187 inch O.D x 0.015 inch wall thickness), is used in antipersonnel low metal mines (APLM) simulants. The aluminum tube produces a reliably different frequency or pitch than the ferrous clutter. Only low metal targets, all of which were mine simulants, were investigated with the cross-over technique. The cross-over technique is summarized in Figure 1-3. (Please note that, as is typical with low metal mines and mine simulants, the mine body in yellow is larger than the MD halo in red. The brightness of the red indicates the amplitude of the MD signal.)
Mine body Step 6 CrossOver Alert

Halo MD signal

Step 2 st 1 CrossOver Alert

MD Alerts

Step 4 Lateral Alert

Step 3 Lateral Alert

Step 5 MD Alert That Bisects Steps 3 & 4

Step 1 st 1 MD Alert

Figure 1-3 MD investigation of a low metal target with the cross-over technique. (Not drawn to scale.)

The cross-over technique begins with the point at which the 1st MD alert is heard along the line of advance (Step 1 in the figure). Either a poker chip is placed at this spot or a mental image is made of this point on the ground, a reference point for further action. The operator then picks up the detector head well off the ground, extends the detector head directly across the line of advance, lowers the detector head to 1-2 inches off the ground, draws the detector head back towards the spot of the MD alert of Step 1 until a second MD alert is found (Step 2). Steps 3 and 4 in the diagram can be done in either order. In Steps 3 and 4, the operator identifies an imaginary lateral line running parallel to the line of advance that runs at right angles and bisects 15

an imaginary line between Steps 1 and 2. The operator then finds the two points upon the imaginary lateral line at which an MD signal first appears moving the detector head from the right (Step 3) and left (Step 4) as shown in the figure. In Steps 5 and 6, the operator bisects the imaginary line running between the MD alert points of Steps 3 and 4 (drawn in the figure) with a vertical imaginary line (drawn in the figure) whose end points are at the MD alert points of this vertical line. Where these imaginary lines cross is the target center. GPR Technique Target Identification Once a proper target center has been established, the target is identified with the GPR. Proper GPR technique, called GPR short sweep, utilizes a normal sweeping motion with respect to speed and position, but is traversed only within the targets MD footprint. In the theory of operation of the AN/PSS-14, the GPR is used to identify the target in the ground. Mines and mine simulants produce GPR beeps from the EU. On the other hand, many commonly found metal objects, e.g., such as nails or other metal fragments, are, in theory, small enough that the GPR will not detect them, thus allowing an operator to discriminate land mines (or land mine simulants) from clutter based upon GPR beeps from the EU. The criteria for identifying targets with the GPR were as follows. If a target produced no GPR beeps, then the target was clutter. If a target produced GPR beeps, the operator had to determine if the GPR signal was consistent or intermittent. Consistent GPR indicated a mine (i.e., a mine simulant). Intermittent GPR indicated uncertainty of the target and was marked separately the target was recorded as being found, but not identified. Intermittent GPR may be the result of poor operator GPR short sweep technique (poor technique can induce a GPR alert) or other physical and environmental factors not fully integrated into the EUs algorithms. Target Identification: GPR Technique & Decision Making Proper GPR technique is required for correct indentification of detected and accurately located targets. Execution of the proper GPR technique is simultaneous with the execution of a decision by the AN/PSS-14 operator of classifying targets into different types. Changes in the POI between 2005 and 2006 produced significant increases in the complexity of the decision making process for the AN/PSS-14 operator. Instead of the much easier binary, GPR-no GPR, decision of 2005, the operator had to discriminate three states no GPR, intermittent GPR, and consistent GPR from one another. This decision process allowed the operator to complete the two tasks required of the AN/PSS-14 operator: identify detected mine simulants or mines as mines and identify detected clutter as clutter. The costs of false negatives, i.e., calling mines clutter, is considerably higher than false positives, i.e., calling clutter mines (aka False Alarm Rate (FAR)). However, the higher acquisition and training costs of the AN/PSS-14 over metal detectors are justified by the clutter rejection capabilities of the AN/PSS-14. Therefore, one should expect the properly trained AN/PSS-14 operator to do an excellent job of mine identification and clutter rejection. Method Research Question 16

The current AN/PSS-14 certification Training Program of Instruction (POI) is a forty-hour POI requiring five days to complete (Table 1-1). Can AN/PSS-14 training be condensed without sacrificing operator performance? Can AN/PSS-14 certification be achieved using a four-day POI (Table 1-2)? Research Design The study was conducted over two years and the results were compared within and between years. Policies for the protection of human subjects as prescribed in Army Regulation 70-25 and the Department of Health and Human Services were adhered to in the research protocol and conduct of this research. Research Participants/Trainees Twelve male high school graduates between the ages of 17-19 were used in the 2006 trial (fiveday POI). Eighteen male high school students between the ages of 17-19 were used in the 2007 trial (four-day POI). Research Apparatus and Training Site Training was conducted at the training facility of the Lincoln University Land Mine Detection Research Center. This training facilty has three components: a classroom complex, a detector sweep training area adjacent to the classroom complex, and a tactical training area. The classroom complex and the adjacent sweep training area were on loan from other Lincoln University units. The Tactical Training Area is dedicated to the Land Mine Detection Research Center. See Hancock (2006) for a more complete description of the Tactical Training Area. The Tactical Training Area consisted of nineteen 1.5 M x 19 M tactical training lanes and six 2 M x 5 M sterile sand pits. Eighteen of these training lanes had been employed by Hancock (2006) for training. Hancock rotated RPs across the 18 training lanes in a manner that ensured that no RP encountered his Exit Test lane during training. The 18 lanes consisted of 9 low clutter lanes and 9 high clutter lanes, used for training. A specified high clutter training lane was used for exit testing only. Although constructed in a manner similar to other high clutter lanes, the specified exit test lane was both flatter and less grass covered than any of the high clutter, training lanes. The same test lane was used during both years of the study. Tactical lanes were 1.5 M wide by 19 M long. Each lane end consisted of a 2 M long sterile area in which the AN/PSS-14 could be ground balanced and the GPR trained. The middle 1.5 M wide x 15 M long section was divided into 10 cells. Each 1.5 M x 1.5 M cell was mapped to a 15cm grid using string lines attached to a 5cm diameter PVC pipe frame designed to fit over cell stakes. The stakes reduced variability by squaring the grid frame on the cell. All targets, both mine simulants and clutter, were placed in cells such that their centers were at a crossing of two lines of the grid at grid coordinates only. All lanes had 9 mine simulants (SIMs): two SIM 6s, one SIM 9, SIM 12, SIM 20, SIM 25, SIM 30, M15 simulant, and M16 simulant. SIMs were arranged with no more than one SIM per cell, 17

leaving one of the 10 tactical cells without a SIM. In each lane, four or five psuedo-randomly selected simulants were placed in proximity to a single piece of clutter such that the MD halos overlapped or adjoined one another. Low Clutter Lanes and High Clutter Lanes had 15 and 30 pieces of clutter, respectively, placed at specific locations. Clutter consisted of the following items: M16 brass (i.e, an expended M16 round), M-60 link, 4 inch (10 cm) barb-wire segment placed in a horizontal alignment with a barb in the middle, 12 D nail, and roofing tack. The lanes are periodically checked with a metal detecter for the appearance of natural clutter and if found, natural clutter is removed. All targets were carefully buried precisely at grid coordinates at fixed distances below the lane surface. All clutter was buried 1 inch (2.5 cm) below the soil surface. The SIM 6s, 9, and 12 were buried 1 inch (2.5 cm) (to top of target) below surface. The SIM 20, 25, and 25 were 2.5 inches (6.25 cm) to top of target below surface. The M16 SIM was 2 inches and the M15 SIM was 3 inches (7.62 cm) to top of target below surface. A grass cover was established and maintained above the lanes. Prior to training/testing, lanes are mowed to a grass height of approximately 2 inches (5 cm). Table 1-1 AN/PSS-14 Certification Training Program of Instruction (POI) (Five-day POI)
Activity Day 1 Training Theory of Operation for the AN/PSS-14 AN/PSS-14 Start-Up & Mine Sweeping Technique Mine/Clutter Footprints Demonstration Mine/Clutter MD & GPR Footprint Practice Day 2 Training Review of AN/PSS-14 Start-Up & Calibration Mine Sweeping Practice Completion of Two Low Clutter Lanes Day 3 Training Mine Sweeping Practice Completion of Two Low Clutter Lanes Day 4 Training Training Area Operator Chipping Training Area 2 Participants to 1 AN/PSS-14 Observer Chipping For Operator Class Room Instruction Training Area Demonstration in Training Area Selected Cells of Training Lanes Location & Notes

18

Mine Sweeping Practice Completion of Two High Clutter Lanes

Training Area Operator Chipping

Day 5 Certification Testing Classroom & Training Area Mine Sweeping Exit Test Participants Individually Scheduled

Table 1-2 AN/PSS-14 Certification Training Program of Instruction (four-day POI)

Activity Day 1 Training Theory of Operation for the AN/PSS-14 AN/PSS-14 Start-Up & Mine Sweeping Technique Mine/Clutter Footprints Demonstration Mine/Clutter MD & GPR Footprint Practice Day 2 Training Review of AN/PSS-14 Start-Up & Calibration Mine Sweeping Practice Completion of Two Low Clutter Lanes Day 3 Training Mine Sweeping Practice Completion of Two Low Clutter Lanes Day 4 Training Classroom & Training Area Mine Sweeping Exit Test Participants Individually Scheduled Training Area Operator Chipping Training Area 2 Participants to 1 AN/PSS-14 Observer Chipping For Operator Class Room Instruction Training Area Demonstration in Training Area Selected Cells of Training Lanes Location & Notes

19

Results Data Collected RPs were scored on the number of targets found out of forty possible targets (9 mine simulants and 31 clutter targets), the number of mine targets correctly identified, and the number of clutter targets correctly identified (clutter rejection). Targets marked as having intermittant GPR were included in the scoring. In 2006, 473 of 480 possible targets were detected, in 2007, 711 of 720 possible targets were detected. The results were strongly influenced by intermittant GPR alerts. When targets marked as producing intermittant GPR alerts are included in the analysis there is no statistically significant difference between the two POI lengths for total targets found, total targets correctly identified, mine targets correctly identified, clutter targets correctly identified, or false alarm rate (Table 13). Trends in the means for each of the experimental variables tend to favor the five-day POI when intermittant targets are included. When the intermittant GPR targets were excluded from the analysis, the results were completely different. In 2006, 279 of 473 (59%) identified targets were marked as intermittant. In 2007, 346 of 711 (49%) identified targets were marked as intermittent. The adjusted mean results for total targets correctly identified, clutter targets correctly identified, and false alarm rate are all hightly significant and favor the four-day POI (Table 1-4). Discussion The study was conducted over a two-year period; the five-day POI was applied in 2006 and the four-day POI in 2007. Twelve research participants (RP) were included in the 2006 interation and eighteen RPs were included in the 2007 iteration. The five-day POI used closely resembled the standard five-day AN/PSS-14 POI in use by the U.S. Army. The AN/PSS-14s ground penetrating radar is designed to emit an audio alarm when it is swept over a mine target. The binary decision mine versus no mine is meant to reduce the time required to clear minefields by distinguishing mines from the other metallic artifacts (clutter). As the operator moves the detector head over the target in the GPR mode in a left-right-left sweeping action (known as the GPR Short Sweep), repeated GPR alerts should be emitted if the target is a mine, and no alerts should sound if the target is not a mine. Occasionally, solid or good GPR is not emitted from a target. This condition is know as intermittant and any target emitting intermittant GPR (IGPR) must be considered a mine. Thus, IGPR increases the false alarm rate when clutter targets are marked as a mine due to their GPR response. The causes of IGPR can vary and not all causes are fully understood. Operator error is a potential cause of IGPR; Operator induced GPR has been demonstrated to occur when operators do not use proper GPR short sweep technique. Other potential contributors to IGPR are soil type (soil classification), soil moisture, and possibly an interaction of these factors. The salinity of soils, in an interaction with soil moisture, may also cause IGPR. As noted in studies 2 and 4 of this

20

report, position of the detector head and target depth can influence GPR response, thus making IGPR potentially more likely. RPs marked IGPR targets using an orange poker chip to distinguish them from clutter (white chip) or mines (red chip). Orange chips were recorded as clutter or mines based on the respective target they marked and the data analyzed (Table 1-3), clutter identification was not significantly different when the two POI lengths are compared. However, treating all orange chips as mines and analyzing the data revealed that the four-day POI group had a significantly higher mean value for correctly identifying targets and a higher rate of clutter identification and a significantly lower false alarm rate than the five-day POI group (Table1-4). This suggests that individuals in the four-day POI group were better able to perform the GPR short sweep to reduce IGPR occurences, or that environmental conditions were such that conditions favoring IGPR were greater in 2006 than 2007. Lately, it cannot be overlooked that RPs may have been more descerning in 2007 regarding declaring a IGPR target than were 2006s RPs. Yet the percentage of targets marked as IGPR is cause for concern. Operationally, IGPR declarations should be treated as mine targets; however IGPR increases the false alarm rate (FAR) and increases the duration of mine clearance operations. While it is encouraging to see the significance of the IGPR adjusted results favoring the four-day POI, caution must be exercised. The small sample sizes (12 and 18, for 2006 and 2007, respectively) make proclamations of the effectiveness of the four-day POI on clutter rejection overly aggressive. It does appear that the four-day POI did not adversly affect total targets found, correct identification of targets, or clutter rejection, regardless of the inclusion or exclusion of IGPR data. The results do highlight the effect of IGPR on operator decision-making and on the FAR. Clearly, focusing GPR training on the operator developing a good consistent GPR short sweep technique is a key factor in reducing the potential for operator induced IGPR. Changes in AN/PSS-14 training and operation doctrine may be needed to reduce IGPR and FAR incidence. Retraining the detector periodically may be an option to reduce IGPR if environmental conditions change as the lane sweeping progresses. Audio footprinting of targets could potentially aid in operator decision making and reduce FAR. Lastly, focusing on the development of excellent GPR short-sweep technique could reduce IGPR decisions and thus the FAR. More research is needed in the area of IGPR incidence reduction techniques, but operator awareness and reaction to changes in the environment-soil moisture, soil testure, temperature, or vegetation conditions and retraining the detector to the changed environment may result in lower IGPR incedence. In conclusion, the effects of POI on clutter rejection observed in this study suggests that statistifactory target detection and clutter rejection can be achieved through a modified four-day POI versus the standard five-day POI. Further, IGPR exerted a considerable influence on the results of the study. The causes of IGPR must be further studied and an understanding of their effest on GPR must be developed. A larger scale study, in excess of 100 RPs per POI group, should be conducted to validate the results of this study.

21

Table 1-3 Mean values of targets found, by targets correctly identified, and false alarm rate by POI length when intermittent GPR targets included. Five-day POI Total Targets Found Total Targets Correctly Identified Mine Targets Correctly Identified (out of 9 possible) Clutter Targets Correctly Identified (out of 31 possible) False Alarm Rate 39.42 33.58 Four-day POI 39.50 30.44

7.67 25.92

7.39 23.06

.1505

.2509

Data were analyzed using a two-tailed t-test. Means shown are not significantly different for POI length at a 0.05 probability of error.

Table 1-4 Mean values of targets found, targets correctly identified, and false alarm rate by POI length when intermittent GPR targets excluded. Total Targets Correctly Identified Clutter Targets Correctly Identified (out of 31 possible) Adjusted False Alarm Rate Five-day POI 16.176 8.50b Four-day POI 20.28a 12.89a Significance 0.011 0.013

0.7124b

0.5789a

0.017

Data were analyzed using a two-tailed t-test. Means with different letters are signigicantly different.

22

Study 2 Studies of AN/PSS-14 Operational Expertise and Training Development Support

23

Introduction Problem Statement Gaps in understanding critical operational factors that affect AN/PSS-14 performance impede efforts to maximize the effectiveness and efficiency of training for operators of this equipment and thus maximize this systems contribution to the Armys countermine capability. A more detailed understanding of the most successful operation of the system is needed to provide training designers, trainers, and trainees with needed knowledge about operational requirements that produce the highest performance levels. Understanding the AN/PSS-14 GPR signals and how operators should interpret them are particularly important needs because these signals are the basis for making mine/no mine decisions. Objectives The prime objectives of the proposed research were to develop knowledge and understanding of expert operation of the AN/PSS-14, apply this understanding to modify new operator training to improve its efficacy, and support empirical assessment of the proposed changes to the AN/PSS14 training program. Achieving the first of these objectives involves a description of the thought processes, strategies, and system operations employed by the most proficient operators of the AN/PSS-14 that enable them to achieve the highest known levels of land mine detection performance and clutter discrimination. To achieve these ends, observational field studies were planned to describe the manipulations of the AN/PSS-14 and the knowledge, perceptual information, and cognitive strategies that enable the best operators to detect buried landmines, especially those with signalevoking material and debris (aka, clutter) adjacent to them, and discriminate mines from nontargeted clutter with maximal accuracy. A key resource in executing this work was the SuperSMS, which was developed to precisely measure movements of the sensor head and their relation to fixed target locations under field testing conditions. Initial efforts were to focus on identifying the factors that yield the most reliable, valid, and useful information from the systems GPR channel in the execution of GPR short sweep. The latter function, when properly executed, contributes heavily to accurate decisions about the nature and location of buried objects -- both mine targets and clutter. Detailed investigation of how experts execute this function involves documenting the range of values on specific kinematic parameters of system operation shown by the best-performing AN/PSS-14 operators and inferring the strategic thinking that guides their operation of the system as well as how the resulting responses of the AN/PSS-14 are interpreted. The goal was to identify valid and objective information about the proper technique for this important function that can be given to trainers and trainees for their use -- information that is currently lacking. This information should include the ranges on operational dimensions such as sensor head velocity, head height above the ground surface, and head orientation (roll and 24

pitch) whose violation produces degradation in system performance. Both trainers and trainees need such well-specified training goals, which constitute critical reference points for assessing operators learning and performance. Because the GPR signal processing algorithms incorporate a learning component and incorporate input from the metal detection subsystem, understanding how experts manipulate the AN/PSS-14 to produce the responses they get in GPR short sweep involves examining their use of the system in executing tasks prior to execution of short sweep over specific targets. This broadens the objective to investigating expert technique on functions such as sensor calibration operations, search sweep, and investigation of targets with the metal detection subsystem. A second related initiative is to compare and contrast the findings of the above effort describing expert techniques to similar findings produced by investigation of the techniques employed by expert operators when engaged in using the AN/PSS-14 to detect demilitarized (demilled) mines, as opposed to the simulated mine targets used for training new operators. Reported differences in the AN/PSS-14 response to demilled and simulated mine targets and the likelihood of corresponding adaptation to such differences by experts raises questions about the generality of the techniques they apply to detection of simulants in training environments. The second main objective was to compare and contrast the findings of the above effort to similar findings produced by a detailed investigation of the performance and techniques applied by new trainees. The third objective was to interpret the results of the preceding efforts and applying them to training design. This will involve determining (1) components of current AN/PSS-14 training whose modification is likely to improve and facilitate operator learning and (2) determine how best to modify these components. The final objective is to support evaluation of the training program modified as anticipated above. Support will involve establishing how specific elements of training should be changed, how they should be communicated to trainees, procedures for training trainers to deliver new material, and procedures for evaluating trainers delivery of the new material(s) in operator training. Background Several studies show that the technologically revolutionary prototype handheld stand-off mine detection systems (HSTAMIDS) prototype and its modified descendant, the nowfielded AN/PSS-14, perform at desirably high levels (Maurer, 2003; Santiago, Locke, & Reidy, 2004; Staszewski, 2001), provided that its operators possess necessary levels of operational skill. Thus, the best possible training is needed to maximize the detection capability of this system, the US Armys countermine capability, and the return on the considerable investment in the development and acquisition of this system. Improved training 25

will also minimize the hazards operators encounter in using the system in countermine operations. Motivation for this effort stems from a field assessment of AN/PSS-14 performance conducted in Afghanistan in the spring of 2003 to address issues that arose about its capabilities and use in the operational environment (Maurer, 2003). Findings showed that the AN/PSS-14 was performing up to it high expectations under operational conditions, subject to the caveat given above -- "provided well-trained soldiers." Two observations supported this qualification. First, in-theater testing showed that the majority of operators' techniques were deficient, resulting in dangerously low detection rates (73.6%) following at least a month without system use. Second, refresher training delivered immediately afterward restored performance to outstanding levels (98.6%) as well as increased the operators' confidence in their detection capabilities with the AN/PSS-14. The implication was that operators' skills had deteriorated in the interval between NET and operational use of the AN/PSS-14. A subsequent in-country assessment of operator skills conducted by USAES personnel also showed that soldiers detection rates had fallen substantially below the standard for successful completion of AN/PSS-14 training (Mincey, personal communication, 2005). This decrement was also attributed to the indeterminate interval without practice between soldiers initial training and subsequent in-theater evaluation. Two subsequent studies were conducted at the request of the USAES to investigate the origins of the problem. One examined the adequacy of the New Equipment Training given to operators at the USAES. A second ongoing study examined the effects of extended interval between AN/PSS-14 training and operational use during which no task practice occurred. Findings from both converged to identify training deficiencies, whose origins can be attributed to the absence of information about system operation. They highlight the need to identify specific kinematic parameters and thresholds related to system operation. Key information is needed by training designers, trainers, and trainees to support trainees effective and efficient learning. The information pertains to specific operational functions and need to specify (1) highly effective (contrasted with less effective) movements of the system, (2) the strategic thinking and goals that these movements serve, and (3) the interpretation given to the output signals that these movements produce, both in the presence of mine targets and clutter. AN/PSS-14 Performance and Training: Research Issues and Investigation Scope A chronic gap in understanding the performance of the AN/PSS-14 limits USAES ability to train operators to the level of proficiency desired and needed. In particular, specification of the kinematic envelope for movements of the system performed with the goal of gathering information via the GPR sensor and algorithms has been lacking. That is, the range over which system movement parameters such as sensor head velocity, head height, orientation to the ground surface (i.e., pitch, roll), spatial extent, and position/trajectory of the sensor head relative to a targets body, can vary and still yield reliable and valid GPR output signals has never been specified. As a result, trainers cannot give trainees clear movement goals, nor can they diagnose 26

and remediate trainees operation of the system to make it produce the information needed for the best possible mine declaration decisions. Our strategy for obtaining this information starts with documenting in detail the kinematic parameters that the best performing AN/PSS-14 operator produces and characterizing the quality and quantity of reliable and valid GPR signals obtained from the range of targets planned for study. Because the experts typically executes multiple swings of the system over a yetunidentified source of MD signals to obtain GPR responses, and some degree of variation is expected for each, the kinematics of swing that produce the expected system response to given targets will be contrasted with those that dont obtain the expected responses to empirically define the dimensional thresholds that distinguish good GPR short sweeps from bad ones. The functional features of the AN/PSS-14 that are related to its generation of GPR signals extend and complicate the search for factors that either constitute effective GPR short sweep or else compromise it. Because learning algorithms are employed to filter ambient terrain irregularities and GPR signal output is conditioned upon the detection of conductive material with the MD sensor field, factors that affect these functions can be expected to influence GPR output. Examples of such factors are the quality of the initial calibration of the MD and GPR sensors and the kinematics of the sensor head in cross-lane search sweep (during which terrain learning occurs). Thus, a comprehensive and detailed examination of all detection activities prior to a particular target encounter has to be conducted to explain system behavior in that encounter. Another significant feature of USAES AN/PSS-14 training that affects the scope of inquiry is the targets used for training. Because the use of actual mine targets is cost prohibitive and also constrained by the limited availability of an adequate supply of safe and realistic mine facsimiles, that is, demilitarized mines, mine simulants are used as targets for operator training. Because of physical differences between mine simulants and anecdotal evidence for differential AN/PSS-14 responses to demilled mine and simulants, it is imperative that results obtained from simulants be compared to observations of the behaviors and thinking (esp. signal interpretation) that characterize the experts encounters with the more realistic, demilled targets and environments in which they are located. Results This project failed largely to achieve the above objectives. When it was evident that the original objectives could not be achieved, alternative plans for working toward the projects objective-albeit reduced in scope--were formulated and executed. Results yielded new and somewhat surprising information about the performance of the AN/PSS-14.The following two sections describe (a) the efforts made to achieve project goals and their results and (b) an alternative study aimed at addressing the stated research problem and a summary of its findings. Development and Testing of Supporting Instrumentation. This project's investigations depended crucially on equipment for data collection that was to be developed and furnished under a separate contract between Lincoln University and Carnegie Mellon Universitys National Robotics Engineering Consortium (NREC). This equipment was to be an enhanced version of the Sweep Monitoring System (Herman & Inglesias, 1999; Herman McMahill & Kantor, 2000) developed as a training technology for AN/PSS-14 Training. The SMS uses computer-based 27

optical tracking of the AN/PSS-14 sensor head as it is moved about in training or performance testing exercises. The enhanced version was designed specifically for research purposes and was to have hardware and software that extended the functionality of the SMS to yield precise measures of sensor head movements in three planes and relative to fixed locations of known buried target items during exercises in the field. This functionality was never achieved, despite efforts to carry out multiple field tests of the S-SMS measurement capabilities. Results repeated showed that this equipment never achieved acceptable levels of reliability and validity, even under favorably restricted conditions of observation. These circumstances denied this project the measurement tool on which it depended despite efforts of the PI and his staff to support and facilitate the development of this equipment by designing and executing multiple field tests of the equipment aimed at empirically assessing it measurement performance. GPR Response as a Function of GPR Short Sweep Accuracy. What is the degree of spatial precision required of operators to use the Army/Navy Portable Special Search-14 to maximally exploit its landmine detection and discrimination capabilities and to accomplish countermine missions safely and effectively with this equipment? Answers to this question hold implications not only for the efficacy, efficiency, safety of countermine operations and the missions they support, but also for training operators of this device and assessing their proficiencies. In the context of operator training, training effectiveness benefits when operators and trainers have clear goals and performance standards with which to assess learning and skill. The successful development and deployment of the man portable AN/PSS-14 landmine detection system represents a significant technological advance as well as leap forward in mine detection capability. It is the first device to successfully employ two distinct sensor systems, electromagnetic induction (EMI or MD), ground-penetrating radar (GPR), and signal processing algorithms that fuse the information these sensors provide. Test results (Santiago, Locke, & Reidy, 2004) show impressive detection performance, especially for landmines with low metallic content, and often referred to as undetectable. Favorable field reports of AN/PSS-14s capabilities followed use of the system in live military operations (82nd Airborne PR) and the striking test results produced by investigations of its capability as a tool for humanitarian demining (HD AN/PSS-14 test, SPIE 2007). Contribution of Operator Training and Skill. The capability that the technological sophisticated AN/PSS-14 offers, however, come at the price of operational complexity. This complexity is reflected as a manifold increase in operator training time relative to the time allotted for training Army operators of the legacy equipment, the AN/PSS-12, which the AN/PSS-14 replaced. Programmatic testing of the AN/PSS-14 prototype, the Handheld Stand-off Mine Detection System (HSTAMIDS), showed the significant contribution that operator training and skill made to system performance (Guckert, 2001; Staszewski & Davison, 2002). Accordingly, the Armys policy for distribution of the AN/PSS-14 mandated that equipment be distributed as a package with its training program. As effective as current AN/PSS-14 operator training is, gaps in knowledge about system performance in the hands of its users exist (Schweitzer, et al., 2006). One is related to relatively large expert-novice differences observed in mine clutter-discrimination performance.

28

A pilot study examining both detection and discrimination of training targets at a site containing AN/PSS-14 training targets and known buried clutter objects (M16 brass, barb wire, barbs, tacks, nails, and M60 chain clips) showed that the most expert user of the AN-PSS-14 achieved a 100% detection rate and near perfect target-clutter discrimination. Although current AN/PSS-14 training produces operators capable of similarly high detection performance, high false-alarm rates, that is, instances in which clutter objects are identified as mines are much higher. One plausible reason for this expert-novice difference in discrimination capability may be the spatial precision with which expert and relatively inexperienced operators perform investigations of potential targets, particularly the phase of investigation that involves an operators generation and interpretation of GPR sensor responses. Here, investigations refer to the procedures operators use once they have encountered an alerting signal in the course of sweeping a lane for targets. The following section provides background on trained investigation procedures for each of the AN/PSS-14s sensors and relates their outcomes to the issues of spatial accuracy and target-clutter decisions. Target Investigation and Mine/No mine Decisions. Investigations are performed in two phases. The first phase involves an operators interrogation of the region where an alert signal has occurred. The operator moves the sensor head over the region with sensor head using motions intended to define the perimeter of a region where the EMI sensor responds continuously. This region is referenced as the MD (Metal Detector) Halo. The linkage of locations where onsets or offsets of MD response signals occur to form a continuous contour defines its perimeter. In the second phase of investigation the operators attention focuses on the output of the GPR system obtained within the MD halo at the point presumed to be the target center. So, in the first step of this phase the operator identifies the point within the MD halo where the targets center should be if a mine is present. Trainees are instructed to infer this point from the MD halos geometry using the primary axes (sideways and forward) of halo symmetry. Their point of intersection identifies this location. A heuristic procedure for validating this inference involves listening to the MD output that results from motions of the detector head about that point and assessing whether the peak MD response occurs at that point. The key operation involves accurately locating the MD halo center and performing side-to-side movements over it, ensuring that the region of the detector head where GPR sensitivity is maximal (the GPR Sweet Spot passes back and forth directly over the halo center. The consistency of the GPR produced by the GPR short sweeps is monitored and interpreted to make a decision that either infers the presence of a mine target or else the presence of clutter-any object or material that is not a mine, but produces GPR signals. Operators decide whether the signals produced in a particular position are those of a mine or clutter based on these signals. The decision rule given to trainees for identifying mines states that if short sweeps of the detector head made over the center of a region defined a by preceding investigation with the EMI produced consistent GPR outputs, the operator should infer the presence of a mine. The above mine-no mine decision rule assumes that operators accurately determine the center of the MD halo and accurately sweep the GPR Sweet Spot over this point in executing short

29

sweeps. An important question to answer is What happens to GPR response when these assumptions are not met? What factors could contribute to inaccuracy in locating the correct location for GPR Short Sweep? First, the process of defining the contours of a potential targets MD halo, identifying its primary axes of symmetry (if it is symmetric), and finding the intersection point of those axes is not done with vision of the physical world, but in the minds eye with visual imagery. Because substantial differences in spatial ability and imagery ability are well documented, individuals at the lower portions of these ability dimensions are likely to experience difficulty in executing these cognitive operations--even under relatively benign, clutter-absent conditions. Accurately locating the center point becomes more challenging at all ability levels under conditions in which metallic clutter is adjacent because intersection of the MD halos for the potential target and an adjacent produce asymmetrical MD halos. This overlap of halos also makes the using the auditory backup heuristic more difficult in the investigation of low-metal mines due to overlap of regions where the auditory MD signals occur. Because the physical size of large (e.g., anti-tank) mines should minimize the effects of inaccuracy on GPR response, inaccuracy could have dire consequences in the case of the smallest anti-personnel targets with low metallic content (APLM). Considering the potential challenges to accurately executing GPR Short Sweep, the consequences of inaccuracy merit systematic study. Accordingly, this research investigates the effects of inaccuracy in the location of GPR short sweep trajectories upon GPR output signals produced by the smallest APLM training targets, SIM6s. Factors Affecting GPR Response in GPR Short Sweep. Investigation of GPR response as a function of short sweep accuracy requires controlling other confounding GPR short sweep dynamics that influence GPR response. One is the height of the sensor head above the ground surface (Rotondo & Ayers, 2003). Another inferred from informal observations and training material is maintenance of a GPR short sweep that keeps the plane of the sensor head parallel to the ground surface over sweep trajectory. Still another inferred from both training materials and informal observation is the length of short sweep; if the trajectory of the search head carries it beyond the boundaries of a candidate targets MD halo, GPR response is affected. Sweep velocity, state of system memory, and the operators setting of GPR sensitivity are additional aspects of AN/PSS-14 operation that affect GPR response and controlled as described in the Method section that follows. An additional factor whose potential influence on GPR response was examined is detector units. Anecdotal reports from operators have suggested differences between units. This source of variability in response was tested by collecting data from two units. Method Design. A factorial design was used. The factors and the number of levels within each were: Targets (4) x Distance (11) x Detectors (2) x Replications (2) x Heights (3) x Number of Short Sweeps (5) x Sweep Direction (2). The design produced a total of 5280 individual sweeps. The distance variable reflected short sweeps performed at either proper position (distance=0) with the center of the GPR Sweet Spot passing over a targets center. Five offsets ahead of the target in 30

the direction of operator heading were selected and listed in Table 2-1 and illustrated in Figures 2-1 2-6. Measurements were also taken at the same offsets past the target.

Table 2-1 Sweet Spot Offset Offset Position 0 1 2 3 4 5 Distance (cm.) 0.0 0.81 3.81 6.35 10.64 13.64 Measurement Points Sweet Spot center over target center Sweet Spot edge over target center Sweet Spot tangential to target edge Sweet Spot edge 2.5cm from target edge Edge of sensor head over target center Edge of sensor head tangential to target edge

The three sensor head heights, measured as the distance from the bottom surface of the sensor head to the top surface of the grass covering the targets, were 0, 0.635, and 1.27 cm. Each measurement trial consisted of a total of 10 GPR short sweeps in alternating direction. Each trial was performed twice at each distance and height with each detector unit. Operator. An experienced AN/PSS-14 operator performed all activities involving use of the detector units. The operator had successfully completed an identical version of USAES AN/PSS14 training in 2005 and had met operator certification requirements (not only in testing immediately following training, but also again after several months without further practice). Intermittent use following the latter test maintained his operational skills. Apparatus. Two AN/PSS-14s, one loaned to the Lincoln Universitys Landmine Detection Center by the manufacturer and one on loan to University of Missouri for Science and Technology were used. The detectors used bore serial numbers 4100039 and 90083. A detector unit harness was designed and fabricated for this experiment to counter the threat of confounds that could influence GPR response and to maximize the spatial precision of short sweep trajectories. The two-piece harness, consisting of an adjustable frame or track that could be fixed at different distance positions on a base, was made of acrylic and is illustrated in Figure 2-7 along with its dimensions. The track supported the AN/PSS-14 sensor head during sweeps to ensure, precisely located trajectories, constant height, and proper sensor surface orientation. The open regions of the base and track allowed for unimpeded GPR transmission and reception over short sweep trajectories. Testing was conducted prior to data collection both 31

over targets and over clear ground to ensure that neither artifactual GPR responses were caused by the harness nor did the harness dilute GPR response over targets representative of those used in this study. The harness is illustrated in Figure 2-7. A digital camera mounted on a tripod was used to record video of each trial. Its audio recording capability was used to record GPR outputs during trials via an open audio microphone attached to the speaker of the AN/PSS-14 in use. Target Materials. Four SIM6 targets located in the training test lanes at the Lincoln University Landmine Detection Center Training Facility were used. Screening procedures were performed prior to target selection to ensure that (a) GPR response in short sweeps produced highly consistent or solid GPR responses, (b) GPR response at position 0 over each target was equivalent when collected both using the brace and without it and (c) that the area adjacent to targets where data collection would occur were free of either metallic or GPR clutter. Procedures After positioning the sweep harness above each target so that it was centered in the middle of the brace at the 0 position, one of the two detectors was selected according to a prescribed order and prepared for operation according to manufacturers procedures. Trials started with the detector unit on the left hand side of the harness and with the first sweep moving it to the right. The movement direction was then reversed with the right-left alternation repeated a total of 5 sweeps in each direction. Prior to the next trial, the operator performed a noise cancel, metal detector ground balance, retrained the system in a clear area immediately adjacent to the target for 12 sweeps at approximately the velocity used for short sweeps, performed another noise-cancel, and then proceeded with the next trial. Each trial was videotaped and the detector units outputs were captured on the video recording units audio track for later scoring. Scoring Three scorers independently reviewed all videotaped trials using video analysis software. For each sweep of the detector head, each scored the GPR signals that occurred by listing either a 0, 1, or 2 for each sweep, with 0 signifying no outputs, 1 signifying a single output signal, and 2 a double output signal. Scoring was reliable. The mean product moment correlation between the scores for each pair of scorers was 0.94, with individual coefficients for each pair of scorers differing by no more than 0.01. Mean trial score agreement was 0.95% with 1% variation among the three pairs of consistency measures. Results Figures 2-2 show the proportion of individual sweeps on which either one or two GPR outputs occur as a function of Sweet Spot-Target offset, head height, and sweep directions separately for each of the four SIM6 targets. Exploratory analyses were conducted to estimate the relative contribution of variables manipulated in this study to explaining the complex pattern of GPR responses. A modeling approach was employed and limited to the positions approaching position 0, or the position in which the most sensitive region of the GPR sensor passes directly over the center of a target. 32

A stepwise logistic regression was first run using only the variables manipulated in this study to (a) assess relative predictive values of these variables and (b) to assess how well main effects attributable to these variables fit the data to establish a baseline level of prediction. In this model, the four targets are dummy coded creating three dummy variables TargetD1, TargetD2, and TargetD3. Results are shown in Table 2-2. Table 2-2 Main Effects Model Testing Global Null Hypothesis: BETA=0 Test Chi-Square DF Pr > ChiSq

Likelihood Ratio Score Wald

1850.5614 1553.3449 1232.8568

7 7 7

<.0001 <.0001 <.0001

Percent Concordant 78.8

Percent Discordant 21.0

c 0.789

Summary of Stepwise Selection

Variable Step Entered DF

Entry Order

Wald Chi-Square Pr > ChiSq

1 2 3 4 5 6 7

Offset Distance TargetD2 TargetD1 TargetD3 Sweep Direction Head Height Detector Unit

1 1 1 1 1 1 1

1 2 3 4 5 6 7

1051.9684 349.2794 68.4984 78.9580 56.7350 11.2254 7.3441 33

<.0001 <.0001 <.0001 <.0001 <.0001 0.0008 0.0067

Classification Table Correct Prob Level Event 0.300 6517 0.320 6514 0.340 6507 0.360 6464 0.380 6425 0.400 6410 0.420 6349 0.440 6262 0.460 6132 0.480 6049 0.500 5966 0.520 5821 0.540 5703 0.560 5617 0.580 5473 0.600 5378 0.620 5181 0.640 5045 0.660 4951 0.680 4779 NonIncorrect NonPercentages Sensi- Speci- False False tivity ficity POS 0.0 24.6 . NEG

Event Event Event Correct 0 2123 0 3 10 53 92 107 168 255 385 468 551 696 814 900

75.4 100.0 75.7 100.0 76.2 76.0 76.1 77.1 76.7 77.2 76.2 76.5 76.9 76.3 75.5 75.8 74.8 74.4 73.0 71.8 72.2 71.1 70.8 34 99.8 99.2 98.6 98.4 97.4 96.1 94.1 92.8 91.5 89.3 87.5 86.2 84.0 82.5 79.5 77.4 76.0 73.3 72.0

27 2096 80 2043 105 2018 148 1975 253 1870 282 1841 405 1718 455 1668 558 1565 679 1444 774 1349 820 1303 930 1193

1.3 24.3 10.0 3.8 23.9 11.1 4.9 23.8 33.5 7.0 23.5 38.3 11.9 22.6 29.7 13.3 22.5 37.3 19.1 21.5 38.6 21.4 21.4 45.8 26.3 20.6 45.6 32.0 19.5 44.8 36.5 18.8 47.3 38.6 18.6 49.8 43.8 17.5 49.2 46.8 17.1 51.3 49.5 16.6 52.0 52.9 16.2 54.3 54.5 16.1 56.0 60.5 14.5 54.9 64.1 13.8 56.1 66.8 13.1 56.2

993 1130 1044 1051 1072 1139 1124 1156 1284 1361 999 1336 967 1472 839 1566 762 1738 705 1822

0.700 4695 1418

0.720 4579 1503 0.740 4476 0.760 4322 0.780 4221 0.800 4036 0.820 3831 1589 1705 1784 1839 1874

620 1938 534 2041 418 2195 339 2296 284 2481 249 2686 189 2866 186 3103 162 3409 148 3755 68 4095 20 4677 0 5287 0 5917 0 6517

70.4 70.2 69.8 69.5 68.0 66.0 64.6 61.9 58.7 54.8 51.8 45.6 38.8 31.5 24.6

70.3 68.7 66.3 64.8 61.9 58.8 56.0 52.4 47.7 42.4 37.2 28.2

70.8 11.9 56.3 74.8 10.7 56.2 80.3 84.0 86.6 88.3 91.1 91.2 92.4 93.0 96.8 99.1 8.8 56.3 7.4 56.3 6.6 57.4 6.1 58.9 4.9 59.7 5.2 61.6 5.0 63.5 5.1 65.5 2.7 66.6 1.1 69.0 0.0 71.3 0.0 73.6 . 75.4

0.840 3651 1934 0.860 3414 0.880 3108 0.900 2762 0.920 2422 0.940 1840 0.960 1230 0.980 1.000 600 0 1937 1961 1975 2055 2103 2123 2123 2123

18.9 100.0 9.2 100.0 0.0 100.0

A second stepwise logistic regression was run entering the variables above and all possible interactions among them. Table 2-3 shows the first seven predictor variables to enter the model and the fit the model produced. As comparison of these models fits shows, the latter model, which includes multiple complex interactions, better explains the observed pattern of GPR responses than the main effects model using only 0-order variables as predictors.

35

Table 2-3 Interactions Model Testing Global Null Hypothesis: BETA=0

Test

Chi-Square

DF 7 7 7

Pr > ChiSq <.0001 <.0001 <.0001 c 0.839

Likelihood Ratio 2695.3326 Score Wald Percent Concordant 82.5 2126.8359 1452.3951

Percent Discordant 14.8

Summary of Stepwise Selection Variable Step Entered DF Entry Order Wald Chi-Square Pr > ChiSq

1 2 3 4 5 6 7

Offset Distance Distance x Height xTargetD3x Unit TgtDum1 DistancexHeightxTargetD1xUnit DistancexTargetD2xSweep Direction DistancexHeightxTargetD1xSweep Direction DistancexHeightxTargetD3xReplication

1 1 1 1 1 1 1

1 2 3 4 5 6 7

1051.9684 618.7339 366.6641 213.7485 176.0137 74.0546 52.9758

<.0001 <.0001 <.0001 <.0001 <.0001 <.0001 <.0001

Classification Table Correct Prob Level NonEvent Event Event 0 2123 Incorrect NonPercentages Sensi- Speci- False False NEG

Event Correct tivity ficity POS 0 6 75.4 100.0 75.6 36 99.9 0.0 24.6 .

0.100 6517 0.120 6511

24 2099

1.1 24.4 20.0

0.140 6511 0.160 6494 0.180 6485 0.200 6465 0.220 6442 0.240 6442 0.260 6442 0.280 6442 0.300 6332 0.320 6302 0.340 6302 0.360 6284 0.380 6284 0.400 6258 0.420 6233 0.440 6214 0.460 6196 0.480 6161 0.500 6108 0.520 5941 0.540 5928 0.560 5913

54 2069 67 2056 88 2035 248 1875 315 1808 315 1808 315 1808 315 1808 565 1558 587 1536 625 1498 625 1498 637 1486 671 1452 736 1387 747 1376 771 1352 814 1309 821 1302 924 1199 941 1182 941 1182

6 23 32 52 75 75 75 75 185 215 215 233 233 259 284 303 321 356 409 576 589 604 649 658

76.0 75. 761 77.7 78.2 78.2 78.2 78.2 79.8 79.7 80.2 80.0 80.1 80.2 80.7 80.6 80.6 80.7 80.2 79.5 79.5 79.3 79.5 79.6 77.6 37

99.9 99.6 99.5 99.2 98.8 98.8 98.8 98.8 97.2 96.7 96.7 96.4 96.4 96.0 95.6 95.4 95.1 94.5 93.7 91.2 91.0 90.7 90.0 89.9 82.8

2.5 24.1 10.0 3.2 24.0 25.6 4.1 23.9 26.7 11.7 22.5 17.3 14.8 21.9 19.2 14.8 21.9 19.2 14.8 21.9 19.2 14.8 21.9 19.2 26.6 19.7 24.7 27.6 19.6 26.8 29.4 19.2 25.6 29.4 19.2 27.2 30.0 19.1 26.8 31.6 18.8 27.8 34.7 18.2 27.8 35.2 18.1 28.9 36.3 17.9 29.4 38.3 17.5 30.4 38.7 17.6 33.3 43.5 16.8 38.4 44.3 16.6 38.5 44.3 16.7 39.1 47.2 16.1 39.3 48.1 15.8 39.2 61.7 13.1 46.1

0.580 5868 1001 1122 0.600 5859 1022 1101 0.620 5397 1310

813 1120

0.640 5317 1350 0.660 5294 1357 0.680 5240 1363 0.700 5155 1398 0.720 4669 1602 0.740 4268 1861 0.760 4268 1861 0.780 4218 1871 0.800 4146 1889 0.820 4146 1889 0.840 4092 1916 0.860 3960 1973 0.880 3306 2009 0.900 3231 2024 0.920 2897 2080 0.940 2786 2086 0.960 2327 2110 0.980 1466 2119 1.000 0 2123

773 1200 766 1223 760 1277 725 1362 521 1848 262 2249 262 2249 252 2299 234 2371 234 2371 207 2425 150 2557 114 3211 99 3286 43 3620 37 3731 13 4190 4 5051 0 6517

77.2 77.0 76.4 75.8 72.6 70.9 70.9 70.5 69.8 69.8 69.5 68.7 61.5 60.8 57.6 56.4 51.4 41.5 24.6

81.6 81.2 80.4 79.1 71.6 65.5 65.5 64.7 63.6 63.6 62.8 60.8 50.7 49.6 44.5 42.7 35.7 22.5

63.6 12.7 47.1 63.9 12.6 47.4 64.2 12.7 48.4 65.9 12.3 49.3 75.5 10.0 53.6 87.7 87.7 88.1 89.0 89.0 90.2 92.9 94.6 95.3 98.0 98.3 99.4 99.8 5.8 54.7 5.8 54.7 5.6 55.1 5.3 55.7 5.3 55.7 4.8 55.9 3.6 56.4 3.3 61.5 3.0 61.9 1.5 63.5 1.3 64.1 0.6 66.5 0.3 70.4 . 75.4

0.0 100.0

Discussion
Figures 2-1 to 2-7 generally show decrements in GPR response consistency with increasing inaccuracy in the positioning of GPR Short Sweep. Both modeling efforts confirm this showing that the distance is the best predictor of consistent GPR response of all the variables measured. This places a premium on instructing operators on the importance of maximizing the precision of their Short Sweep targeting. This, in turn, suggests the importance of sound technique for spatially distributed investigation of alerting signals with the metal detecting sensor, whose results are used to determine the target point for GPR Short Sweep.

38

A surprising finding is that decrements are extraordinarily inconsistent as a function of the direction in which the sensor head deviates from being swept top dead center over the targets. The physical symmetry of the sensor head may predispose operators, as it had this investigator, to expect symmetric sensitivity within its field. The data show that this is clearly not the case, with the rearward portion of the detector being less sensitive to Short Sweep inaccuracy than the leading half. It is unknown why the rearward portion of the sensor head shows greater sensitivity than the leading half. This differential is unfortunate because exploiting this relative advantage in an operational environment would require the operator to work in closer proximity to the mine threats sought. The pattern of GPR response observed, particularly that approaching the target is explained reasonably well by the models developed, however both the unexplained variation in the data and the variation explained by the best-fitting model beg important questions. Starting with the latter, target variability accounts substantially for variation in GPR response after the variability related to Short Sweep inaccuracy is accounted for. It is unclear what underlies these differences, noting that candidates include unmeasured differences between the target simulants components as well as possible unmeasured differences in the soil and vegetation surrounding the targets. Five of the predictors in the better fitting regression model are four-way interactions. The implication is that the explanation of GPR Response that the model produces is an extremely complicated one, involving many conditionals. In other words, predicting GPR response as a function of Short Sweep inaccuracy is highly dependent on the context or circumstances. This complexity poses problems for easy and quick understanding as well as communication, which are important instructional considerations. Some consistency can be found in these complex predictor interactions, however. All five of the interactive predictors include the distance variable, underscoring the role this variable play in explaining GPR response variation. Four predictors incorporate both distance and height in interactive relations, along with one of the dummy target variables. The consistent appearance of these variables suggests that they should be focal points in further investigations of GPR response. Understanding the observed pattern of GPR responses is limited by the absence of potentially measurable variables. Considering that GPR outputs require the detection of conductive material by the metal detector and that its presence can be at a level below that for production of a metal output signal, such sub-threshold metal signals may be influencing GPR response. Such effects offer a plausible, if speculative, account for GPR response measures at position 5 (Distance = 13.64 cm) in Figures A-C. Only instrumentation that allows for such measurement would allow this speculative, but reasonable hypothesis to be tested. Still another unmeasured variable with potential explanatory value is the internal state of the software at the time metal and radar returns register. Because the AN/PSS-14 compensates for relatively local, but consistent soil and terrain variability via ground balancing and GPR training, these adaptations to local conditions may represent another source of response 39

variability. Only a detailed characterization of what these adaptations involve and how such algorithmic adaptation affect processing of inputs of both sensors can their role in explaining GPR variability be assessed. In conclusion, this exploration of the impact of the accuracy of GPR short sweep upon GPR responses has raised far more questions than it answers. In as much as safe and effective operation of the AN/PSS-14 involves understanding its behavior and thorough understanding is critical for effective operator instruction, further investigation of the issues raised by this work is warranted and needed.

Figure 2-1

40

Figure 2-2

Figure 2-3

41

Figure 2-4

42

Figure 2-5

Figure 2-6

43

Figure 2-7 Detector Unit Harness

Figure 2-8 GPR Response Curves for Target A

44

Figure 2-9 GPR Response Curves for Target B

Figure 2-10 GPR Response Curves for Target C

45

Figure 2-11 GPR Response Curves for Target D

46

Study 3 Robotic Mine Sweeper

47

Introduction Description of the Requirements In FY08, Edge Robotics adapted its robotic technologies to build a prototype of a mine sweeping robot. The robot was built according to the following specification: Modular: the robot can be adapted to work with various mine detectors, such as the Minelab F1A4, F3, AN/PSS-12, or AN/PSS-14. In most cases, mounting a new detector only requires a new mount and interface electronic. But in some cases, such as the AN/PSS-14, the interface to it is more complex due to the amount and type of data that the detector generates. 2-Axis Sweeping Arm: the robot is equipped with a 2-axis sweeping arm, allowing the detector to move up and down, and left and right. Since some detector requires a certain sweep rate, the speed of the sweeping arm is programmable in software. On-board positioning system: the robot does not only need to detect a mine, but it also needs to know the location of the detected mines. This is done by using a differential GPS system augmented by an inertial navigation system based on gyros and accelerometers On-board electronic box: because there are always loss and degradation of data over the radio communication link, all the critical controls of the robot is performed on-board. Radio-communication link: the robot needs to be controlled remotely by the remote operator, and the result also needs to be displayed to the operator Power subsystem: the mobile platform and all the payloads need power to operate. This power is supplied by an on-board battery pack equipped with power regulators to provide steady power to everything on the robot. The Robotic Mine Sweeper The robotic mine sweeper is depicted in Figure 3-1 2-AxisManipulatorDifferentialGPSWarning Light & E-Stop ButtonElectronicBoxStereo CameraMine DetectorClutch and Fail-safe E-StopDifferentialDrive MotionBase2-AxisManipulatorDifferentialGPSWarning Light & E-Stop ButtonElectronicBoxStereo CameraMine DetectorClutch and Fail-safe E-Stop2-AxisManipulatorDifferentialGPSWarning Light & E-Stop ButtonElectronicBoxStereo CameraMine DetectorClutch and Fail-safe E-StopDifferentialGPSWarning Light & E-Stop ButtonElectronicBoxStereo CameraMine DetectorClutch and Fail-safe E-StopDifferentialDrive MotionBase It is shown with the Minelab F3 metal detector mounted on the 2-axis manipulator Figures 3-1 and 3-2). The detailed technical specification is as follows:

48

 Driving actuator: Two DC brush motors equipped with a manual clutch and fail-safe brake. The manual clutch can be disengaged if the robot needs to be pushed around manually. The fail-safe brake will engage if power is loss. Chassis: differential drive chassis with front drive motors which are mounted on adjustable front-suspension. The two back wheels are passive caster wheels. 2-Axis sweeping arm: o Pan axis: this axis is actuated by a brushless DC servo motor with a built-in controller. The motor is first coupled to a planetary gearbox. The output of the planetary gearbox is then coupled to the arm using a belt. o Vertical axis: this axis is actuated by a brushless DC servo motor with a built-in controller coupled to a linear gearbox. Perception sensor: o The front bumper has three switches to detect collision with an object o A high resolution stereo camera is used to detect potential obstacle in front of the robot Electronic Payload: o Dual channel brush DC motor controller capable of 60A steady state drive current and 120A peak. o Vehicle controller: a microcontroller and custom FPGA board augmented with a PC-104 CPU board for closed-loop control o Computing boards: Two Core-Duo Mini-ITX board, each equipped with a 2.0GHz CPU and 2GB of RAM o Pose sensors: differential GPS, gyros, accelerometers, and dual wheel encoders o Interface electronic: a custom FPGA board to interface to the detector, GPS, gyros, accelerometers and wheel encoders Power Subsystem: o Primary battery pack: 2x13AH Sealed Lead Acid/SLA batteries o Secondary battery pack: 2x13AH Sealed Lead Acid/SLA batteries o Power regulators: DC-DC converters to generate regulated 5V and 12V DC from the unregulated 24V DC from the battery pack Radio communication: o Hand-held radio remote with dual joystick for driving the robot remotely o 900Mhz radio for communication with the Operator Control Unit Operator Control Unit: o Main OCU: a Panasonic Toughbook TB-19 convertible notebook. This is a rugged notebook with touchscreen and high-brightness screen o Handheld OCU: a hand-held radio remote control Test Results We did a preliminary test of the robot at the Lincoln University land mine training site. We set the system up and tested the robot on several mine lanes. We also did a demonstration on the main campus. 49

Operational Requirements and Start Up Procedures o Base station: the GPS base station which provides the differential GPS correction to the robot needs to be placed on an area with a clear view of the sky. It needs 110V AC power. The unit needs to be turned on and after 10 minutes, it will start transmitting GPS differential correction over the 900MHz radio o Operator Control Unit (OCU): the OCU can be setup up 50-100ft away from the robot. If extended operation is required, the OCU can be powered from 110V AC. o The robot needs to be fully charged by connecting it to the charger for about 2 hours. A fully charged robot will operate for between 1-2 hours depending on operating conditions. Once set up was complete, mine lane sweeping with the robot commenced. Here are a few pictures of the robot sweeping the lanes: And a picture of the demonstration on campus: Based on the tests we did here, we are going to make a few improvements in the future that will further increase the usability of the robot. These improvements include: o Better user interface to allow the operator to interact better with output of the mine sweeper. An example of this includes the ability to zoom in and out on the output of the detector o Easier control of the robot: we will improve the ability to survey and import GPS coordinates of the mine lanes to automate the motion of the robot over the mine lane. o Better positioning: from the result of the test, it seems the GPS based positioning system is sensitive to GPS interference. We will improve the algorithm to make this more robust.

50

Figure 3-1. Robot with Minelab F-3 detector attached.

51

Figure 3-2. Robot with F-3 detector conducting target sweep in a mine lane.

52

Study 4 GPR Alert Signal Reliability

53

Introduction University of Missouri Rolla CEST Team has been collecting audio data from the AN/PSS-14 mine detectors. Audio recordings were collected over VSE mine simulants of various sizes, simulants developed by University of Missouri Rolla, as well as common types of metal clutter. Emphasis was placed on the consistency and behavior of the auditory GPR response from the detector. Research results show that GPR gives off different auditory responses (sometimes no response at all) through a single recording of several passes of the detector head over the target. Variation has also been noted between two different AN/PSS-14 units over the same target, when trained and operated under identical conditions. Clutter was also found to cause the GPR audio signal to sound off in a majority of cases, increasing the false alarm rate (FAR), and instances were observed where no GPR alerts were given over mine simulants. Method An apparatus was developed for the purpose of swinging the detector under controls not possible when the detector is swung by a human. Specific and defined parameters can be established by adjustments to the apparatus and at the same time factors introduced by normal human limitations in performance can be eliminated. For example, the apparatus permits precisely controlling the speed of sweep, head height above the surface, depth of burial of the targets, lateral location of the target relative to the detector head, and a specific method of training the GPR algorithms. Additionally, the apparatus permits precisely repeating conditions for each test. Thus, the system of sweeping the AN/PSS-14 detector enables direct comparison of performance of the same detector repeatedly sweeping over a target in exactly the same conditions. It also enables comparison of performance of different units sweeping over the same target in identical conditions. Digital data collection can then provide an objective evaluation and comparison of performance. The AN/PSS-14 is mounted to and rotates from a short piece of notched PVC pipe that locks into another PVC pipe attached to a pulley underneath the platform. The pulley is rotated by a belt attached to an 12V electric motor that sits away from the detector (approx 31 vertical, 37 horizontal, for a straight line distance of 48 at its nearest point). The detector is operated in a tub filled 12 deep with sand where targets can be buried. Noise cancel and ground balance are done while the electric motor is spinning, but no belt is attached, allowing cancellation of any electrical noise generated by the motor. GPR training is done while no targets are buried, at the swing speed that data will be collected. Adjustments to the GPR sensitivity are made after GPR training is done to eliminate GPR alerts over clean sand. After this process is complete a target is buried in the path of the detector head.

54

Figure 4-1 Testing Setup Data Collection. Audio data is collected by attaching an Edirol R-1 24bit WAVE recorder to the headphone out connector on the detector. A recording of the audio response is created for each target, and then taken back to a PC to be analyzed. Adobe Audition is used to give a visual representation of the metal and GPR response. Results For the past year data have been collected over mine simulants and clutter. The data have shown that variations in GPR response frequently occur. Data have been collected that show times where the GPR auditory signal is not produced over the target. GPR response is sometimes very strong and consistent, at other times targets are almost indistinguishable from GPR signals that appear over soil that has no targets and thus is unexplainable. GPR audio response is indicated on the following visuals by vertical lines above and below the noise floor. Tests were conducted to determine if GPR response can be expected over common pieces of clutter in various orientations: nails, sections of barbed wire, machine gun chain links, M16 brass shell casing, and a roofing tack. The only two cases where clutter did not give any GPR response were the barbed wire and nail placed vertically into the ground. GPR response also showed instances, indicated in the highlighted fields, where the GPR audio alert occurred inconsistently, that is it sounded off on only a portion of passes over the clutter target. This indicates that while nothing has changed with the target, the detector has given a different response than it did at other times over the target.

55

Figure 4-2

GPR audio response over a Sim 25 showing consistent GPR over the target as indicated by the thick vertical lines. The thinner vertical lines occasionally seen are occurring over areas where nothing has been placed.

Figure 4-3

GPR audio response over a Sim 12, brackets above the vertical lines indicate simulant location. GPR audio response is inconsistent over the target, and occurring several times where no targets were placed.

56

Table 4-1 GPR Response To Clutter By Time, Date, and Clutter Orientation
-90084 File R1_001 R1_002 R1_003 R1_004 R1_005 R1_006 R1_007 R1_008 R1_009 R1_010 R1_011 R1_012 R1_013 R1_014 R1_015 R1_016 R1_017 R1_018 R1_019 R1_020 R1_021 R1_022 R1_023 R1_024 R1_025 Clutter Type Barbwire Barbwire Barbwire M16 Brass M16 Brass M16 Brass M16 Brass Link Link Link Link Link Link Link Link Link Link Link Link Tack Tack Nail Nail Nail Nail Orientation Parallel Perpendicular Vertical Parallel Perpendicular Vertical (Primer Up) Vertical (Primer Down) Open End Up & Pointing Out Open End Up & Pointing In Open End Up & Pointing Towards Open End Up & Pointing Away Open End Down & Pointing Out Open End Down & Pointing In Open End Down & Pointing Towards Open End Down & Pointing Away Open End Out & Pointing Down Open End In & Pointing Down Open End Out & Pointing Up Open End In & Pointing Up Parallel Perpendicular Parallel Perpendicular Nail Head Up Nail Head Down 8/14/2006 % GPR 67% 0% 10% 100% 0% 0% 0% 100% 100% 89% 100% 0% 8% 30% 0% 0% 100% 0% 11% 0% 0% 100% 0% 0% 0% 08-15-06 Morning % GPR 100% 0% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 0% 0% 0% 0% 0% 0% 100% 0% 08-15-06 Afternoon % GPR 0% 0% 100% 100% 70% 70% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 20% 20% 0% 0% 0% 30% 08-16-06 Morning % GPR 100% 0% 100% 80% 100% 0% 0% 100% 100% 100% 100% 100% 100% 100% 100% 100% 90% 100% 100% 0% 0% 100% 0% 0% 0% 08-16-06 Afternoon % GPR 100% 0% 100% 100% 100% 0% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 30% 0% 100% 0% 100% 100%

57

Clutter is buried with the center point below the surface of the sand directly below the center of the detector head. Parallel refers to placement of clutter along the 9 oclock to 3 oclock position of the detector head, perpendicular refers to 12 oclock to 6 oclock direction. Vertical refers to the clutter upright in the sand with the topmost point below the surface of the sand.

Head Height and Offset Tests Tests were conducted February-March 2007 to determine what if any effect detector head height and position over the target, a VSE Sim 6, had on GPR response. The detector was trained in the sand tub with no foreign objects in the tub. The simulant was placed with the topmost portion below the surface of the sand. Data were collected at six heights above the surface from three inches inside the center of the detector head to three inches outside the center in one inch increments. Figure 4-4 Target Placement Relative To the Center of the Detector Head

Sim 6 was placed at deep -3, -2, -1, 0, 1, 2, 3 with 0 being the center of the detector head. The large circle indicates the path the detector head was swung on.

Audio recordings were made of the output from the detector for 25 passes at each height for every offset position. During the testing it was recorded when an auditory GPR alert occurred over the target, giving the percentage of total passes with a GPR alert. 58

Initial findings (Table 4-2 and Figure 4-5) indicated greater GPR response at a 1.5 head height and with the target located under the front of the detector head (away from the operator). Subsequent tests (Tables 4-3 and 4-4 and Figures 4-6 and 4-7) were unable to confirm the initial results. The first repeat of the initial test gave no GPR response over the target at any height or offset. Another follow up test was conducted on a separate day and while confirming greatest detection at a head height of 1.5, it directly opposed the earlier data by indicating the rear of the detector head gave greater GPR response. Table 4-2 GPR Response By Detector Head Height and Target (Sim 6) Offset
Height (inches) Offset (inches) -3 -2 -1 0 1 2 3

1.25 0 0 92 0 100 100 100

1.5 65 0 100 100 100 100 100

2.25 24 60 4 100 32 100 100

2.34 4 80 0 100 0 88 50

3.25 0 0 0 0 0 12 0

3.875 0 0 0 0 0 8 10

Data Collected 21 February 2007. Target was buried 1.27cm below sand surface. Percentage of passes with GPR auditory alert at each position relative to simulant.

59

Table 4-3 GPR Response By Detector Head Height and Target (Sim 6) Offset (15 MAR 07) (Burial Depth 1.27cm)
Head Height Above Surface Offset -3 -2 -1 0 1 2 3 100 100 100 100 100 100 0 1 0 0 0 0 0 0 0 1.5 0 0 0 0 0 0 0 2 65 0 80 95 0 0 0 2.5 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0

Data collected 15 MAR 07. Target burial depth 2.27cm. Percentage of passes with GPR auditory alert at each position relative to simulant, depth 1.27cm.

60

Figure 4-5 Percentage of Passes with GPR Alert (21 FEB 07)
80-100 60-80 40-60 20-40 0-20

3 Offset from Center of Detector Head (inches) 1 -1

100 80 60 40 20 0 -3 3.875 3.25 2.34 2.25 1.5 1.25 Percentage of Passes with GPR Alert

Head Height Above Ground (inches)

61

Figure 4-6 Percentage of Passes with GPR Alert (15MAR 07) (Burial Depth 1.27cm)
80-100 60-80 40-60 20-40 0-20

Percentage of Passes with GPR Alert

100 80 60 40 20 0 0 1 1.5 2 Head Height Above Ground (inches) 2.5 3 3 2

-3 -2 -1 0 1 Offset from Center of Detector Head (inches)

Percentage

of GPR audio alerts by detector head position relative to simulant.

62

Table 4-4 GPR Response By Detector Head Height and Target (Sim 6) Offset (15 MAR 07) (Burial Depth 5cm)
Head Height Above Surface Offset -3 -2 -1 0 1 2 3 100 100 100 100 100 100 0 1 0 0 0 0 0 0 0 1.5 0 0 0 0 0 10 0 2 5 0 10 16 0 0 70 2.5 0 0 5 0 0 0 0 3 0 0 0 0 0 0 0

Data collected 15 MAR 07. Target burial depth 5cm. Percentage of passes with GPR auditory alert at each position relative to stimulant, depth 5cm.

Tests with two detectors Tests over UMR made simulants were conducted in early October 2006. They were done over a complete set of simulants with two AN/PSS-14 detectors (Serial No: SERNO-90083, SERNO90084) where data was collected with each detector without disturbing the simulant. The setup and training process was the same for each detector before data collection. The average length of time the GPR audio signal was on over the simulant was compared between detectors and also to the time the detector head was over the target. The results show the 90084 yielded a longer GPR audio response than the 90083 in nearly every pass. Both detectors had GPR audio responses over a greater period of time than the detector head was actually over the simulant. See Figure 4-8 for bar graphs indicating time GPR audio signal is on when generated by passage over each specific target between detectors.

63

Figure 4-7 Percentages of Passes with GPR Alert 80-100 60-80 40-60 20-40 0-20

100 80 Percentage of 60 Passes with 40 20 GPR Alert 0 0 1 2 1.5 2 2.5 3 3

-3 -2 -1 0 1 Offset from Center of Detector Head (inches)

Head Height Above Ground (inches)

An interesting observation was at +3 offset GPR Alerts occurred 70% of the time at a burial depth of 2 inches (5cm), but 0% of the time at a burial depth of inch (1.27cm).

64

Figure 4-8 GPR Auditory Alert Over Target, By Set, By Detector

GPR Auditory Alert Over Target, Set 1
1.4 1.2
1

Seconds

0.8 0.6

SERNO-90084 Head Time Over Target SERNO-90083

0.4 0.2 0 Sim 6 Sim 9 Sim 12 Sim 20 Sim 25 Sim 30

GPR Auditory Alert Over Target, Set 2

1.2000 1.0000

Seconds

0.8000 0.6000 0.4000 0.2000 0.0000 Sim 6 Sim 9 Sim 12 Sim 20 Sim 25 Sim 30

SERNO-90084 Head Time Over Target SERNO-90083

GPR Auditory Alert Over Target, Set 3

1.2000 1.0000

Seconds

0.8000 0.6000 0.4000 0.2000 0.0000 Sim 6 Sim 9 Sim 12 Sim 20 Sim 25 Sim 30

SERNO-90084 Head Time Over Target SERNO-90083

1 GPR audio response in seconds over target with actual head time over target included for reference.

65

In one test the 90084 missed the Sim 6 nine out of ten passes while the 90083 picked it up every time. The 90083 displays several GPR (Figure 4-9) alerts where no targets are present whereas the 90084 will not, this was observed throughout testing despite both being at the same sensitivity setting, six clicks from minimum.

Figure 4-9 Comparison of the GPR Response of two AN/PSS-14 Detectors Over the Same Sim 6 Target
Detector Serial Number -90084 over a Sim 6 Detector Serial Number -90083 over the same Sim 6

Another example of different responses between detectors is shown in Figure 4-10. In this case a Sim 12 was missed twice out of the 11 passes by the 90084 and picked up every time with the 90083.

Figure 4-10 Comparison of the GPR Response of two AN/PSS-14 Detectors Over the Same Sim 12 Target
Detector Serial Number -90084 (Missed 1st and 6th pass) Detector Serial Number -90083

66

Other Observations The GPR may occasionally enter a hypersensitive mode, followed by a period of less sensitivity. This experience is shown in Figure 4-10. The GPR audio signal is represented by the tall and narrow vertical lines. The metal detector audio signal is indicated by the thicker, shorter and consistently repeating lines. These metal detector audio signal lines represent the metal detector audio signal that occurs at each pass over the target as per each revolution of the detector around the tub. In this case the target is a nail at the 6 oclock orientation. Notice the grouped GPR audio signals that occasionally occur when the detector head is traveling over the clear soil away from the nail. When these groups occur over the clear soil, the GPR does not alert over the nail. However in the cases where the GPR does not alert during the pass over the clear soil, a GPR alert is given over the nail.

Figure 4-11 GPR Response Over Nail Placed Perpendicular to Detector Head
GPR is the tall lines, Metal is the thicker but shorter, consistently repeating lines. Notice how when there is significant GPR where no targets exist before the metal, no GPR is obtained over the metal, but when there little or no GPR before the metal, a GPR response occurs.

67

Conclusion The data collected displays inconsistencies in the GPR response of the AN/PSS-14 over targets, inconsistencies occurred by test date target type, time of day, and target orientation. There is also a difference in GPR response between two detectors when they are identically trained and used in identical conditions. GPR response often occurs over pieces of common clutter, reducing the efficiency of mine detection by increasing the amount of targets to be investigated as a mine. Data indicates that head height is a factor in the probability of GPR alerts for a mine with a range of 3.8-5cm above the surface being optimal for mines at the tested 1.27cm and 5cm below the surface (metal signal is strongest closest to the ground). GPR response has been shown to not be equal between the front or back of the detector head, with data indicating that higher detectability will not be at the front or back each time used. Data seems to indicate the detector will not perform the same each and every time it is used.

68

Study 5 The Use of Computer Aided Instruction to Sustain Land Mine Detection Operator Performance1

69

Introduction Studies have shown land mine detector operator performance decreases significantly from 30 to 90 days from the initial training certification event, as measured by the number of targets detected (Hancock, 2006). Proficiency maintenance through the use of simulations is common place in the aviation community and is finding a niche in medical professions to sustain surgical skills (Stefanidis et al., 2005). Sustainment training in land mine detection at periodic intervals is necessary to prevent significant skill erosion which exposes land mine detector operators to increased risks. Training minefields are not always available to support sustainment training; especially for Marines afloat. Computer aided instruction may provide a means to sustain land mine detector operator skill when no mine training area is available. This study was designed to evaluate a computer-aided instruction (CAI) program as a means to develop and sustain land mine detector operator performance versus practice as measured by operator testing using a mine metal detector in a simulated lane. Results are discussed as a function of the number of targets located (out of 40 possible targets) and the accuracy of finding the center of each target. Methods The study used 27 college students (20 male and 7 female) as participants; they were divided into three training methodology groups (control, practice, and computer-based). All participants received a 45-minute class in the operation of the metal detector, followed by a 30-minute demonstration of traveling sweep, investigative, and centering techniques. All participants then practiced traveling sweep, investigative and centering techniques using the metal detector by sweeping portions of selected 1.5 m by 1.5 m cells with known target centers identified by white chips. This process was used to train metal detection technique, the participants would investigate and mark the metallic halo around the target (Figure 5-4). Later, the participants swept three 1.5 m by 1.5 m cells to identify targets and place chips on perceived target centers. The participants swept and identified targets in the first cell, then were provided feedback on their sweeping technique and chip placement. They then swept the remaining two cells, placing chips on perceived target centers, followed by feedback on their techniques and performance. The following day, participants returned and were tested on their abilities to detect targets in a mine lane and to accurately mark the center of the target. Prior to testing, participants received either no practice (Control), practice by sweeping three 15 m by 1.5 m mine lanes with feedback, or CAI training by sweeping three virtual 15 m by 1.5 m lanes. The CAI package, developed by Missouri University of Science and Technology, consisted of software that was loaded onto a laptop computer and a mouse to move a virtual detector head. Participants used the mouseoperated detector head to sweep the virtual lane (traveling sweep technique), investigate identified targets (investigative technique), and position a chip over the center of each target. A Minelab F-3 metal detector was used to detect targets in the practice and testing lanes. US Army approved mine simulants of varying sizes and clutter targets were buried in the practice and test lanes. Targets were positioned in the lanes using a 15 cm by 15 cm grid system; the target centers being located at the intersection of horizontal and vertical gridlines. Each group 70

used the F-3 metal detector to detect simulated or inert anti-personnel (AP) and anti-tank (AT) mines; each type of mine having both high (HM) and low metallic (LM) content mines buried in the 15 m x 1.5 m test lane. Additionally, metallic clutter material consisting of roofing tacks, 12 D common nails, 5.56 mm (M-16) ammunition casings, 7.62 mm (M-60) machine gun ammunition links, and barbed wire were also added to the lanes. A total of 40 identifiable targets were present in each lane. Results were analyzed using one-way analyses of variance and means were separated using Tukeys HSD post-hoc tests ( = 0.05). The research was conducted over four two-day periods at the Lincoln University Land Mine Detection Research Center in Jefferson City, Missouri during March and April 2008. Data were collected on the total number of targets (out of 40 possible targets) correctly located, targets located and chip placed within the allowable distance from the actual target center, and the measured distance from the point each participant marked as the center of the mine to the actual center of the target. Allowable distances for chip placement from actual target center (measured from the intersection of the grid lines to the closest edge of the chip) were established using criteria adopted by the US Army in training soldiers to detect mines using these same mine simulants and clutter types. Chip placement exceeding the allowable distance resulted in the target being scored as a miss. Results Analyses of the raw data showed the resultant scores for targets found and target centers within allowable distance were not normally distributed and skewed to the right. A logarithmic (base 10) transformation was applied to bring the data into a normal distribution for analysis of variance and mean separation. The distribution of the average distance from target center data was normally distributed; thus no transformation was performed on these data. Analysis of the transformed data for targets found showed the mean score of the CAI participants was significantly lower than the practice group (Table 5-1). No significant differences were found between the number of targets found by the control and either the CAI or practice groups (Table 5-2). No significant differences among training methods were identified in either the average distance from target center or the transformed data for targets found within the allowable distance. Further review of the raw data revealed one outlying score among male participants in the CAI group for total targets found; the participant scored ten percent lower on total targets found than the next lowest score. When the score of this participant was removed and an ANOVA of the remaining participants conducted, no significant difference was observed between the CAI and Practice groups. When performance by gender was considered, the mean scores for targets found and targets located within distance of the female participants were lower, and the average distance from target center was slightly higher than the male participants , but neither of these differences was found to be significant (Figures 5-1, 5-3). When just the male participants scores were analyzed using ANOVA, we found that training method did not have a significant impact on targets found, targets located and chips placed within the allowable distance, or the average distance of the chip from target center.

71

Discussion The expected result was that there would be no significant differences between the CAI and Practice groups in terms of targets detected, targets located within allowable distance and average distance from target, but that both of these would be significantly higher than the Control group. The absence of significant differences between the mean scores of the Practice and CAI groups in the three parameters measured when compared to the Control group was a surprise. Targets detected is a measurement of the participants traveling sweep technique and ability to recognize tonal variations emitted from the mine detector which may indicate multiple targets with overlapping metallic halos. Observations made while the participants were negotiating the test lane offer some insight into factors that may have influenced the results. We sensed that the Control group tended to compensate for their lack of practice by taking more time to complete the testing lane. Performing an ANOVA on the time on task data (Table 5-3) did not reveal a significant difference among the training method groups; but it is clear that not only did the Control group spend, on average, more time completing the test lane, their minimum and maximum times were greater than the minimum times for either the CAI or Practice groups and the maximum time for the Practice group. CAI participants tended to have a poorer traveling sweep technique than the Practice group. Three factors may contribute to this observation. The first factor is the CAI group trains on traveling sweep and investigative techniques using a mouse and not a mine detector, hence the motor skills used in training are not the same motor skills used in testing. Next, CAI participants sit during their training, whereas the Practice group trains as they will test on their feet and moving forwards and back, squatting and reaching to place chips. Lastly, the virtual mine lane on the computer screen is not 1.5m wide, it does not have three dimensions with grass clumps impeding the travel of the detector head, nor bare spots in the grass. None of the study participants had used metal detectors previously; hence the use of CAI to conduct initial training for mine detector operators may not adequately develop the techniques (traveling sweep or investigative) necessary to accurately detect and mark targets in an actual minefield. When the results of the analyses of the adjusted data (N=26) for targets detected and the targets located within allowable distance are considered, they suggest that CAI can be effectively used to sustain operator skill levels. Women comprised 25% of the total participants in the study and 33% of the CAI participants. Performance differences by gender were subject to a high degree of variability and small sample size (N=7), thus no statistically significant gender effects were observed. However, of the four lowest scores in targets found, three belonged to women and of these three; two were trained using CAI. This suggests that adaptability to CAI for land mine detector training may be influenced by gender to some degree. It is likely that more factors than gender influence compatibility with CAI training. Comfort with technology and socioeconomic factors may also play a role. Conversely, one woman (Practice group) found all forty targets, one of only four participants to do so in the study. Four women scored in the range of 36-40 targets found; including one woman from the CAI group. Thus, gender should not be accepted as a criterion to exclude women from performing the mine detection task. 72

If there is a negative effect to the use of CAI to sustain mine detector operator skill levels, it might be related to the characteristics of physical and functional fidelity, as described by Alexander et al. (2005). None of the participants had prior mine detection experience and only received a brief familiarization with the detector the day prior to testing. It is possible that some CAI participants did not develop a sense of the tonal output and handling characteristics of the F3 (physical fidelity) during the familiarization exercise and took longer to adjust to the F-3s when transitioning from the CAI during testing, which may have resulted in the lowering of their scores and increasing their average distance from target center. Fidelity further suggests that, regardless of gender, there may be a target audience for which CAI is not an effective means of conducting initial training for mine detector operation. An experimental methodology where CAI participants navigated two virtual lanes followed by one practice lane might have produced better CAI scores. Trained mine detector operators who have a well developed sense of the physical and functional fidelity factors of mine detection should benefit from CAI training designed to sustain their skills when facilities to conduct practice are limited or not available. As Stefanidis et al. (2005) report, periodic training with a simulator sustained the laparoscopic skills developed in a group of medical students to a significantly higher degree above a control group at six months post training. Sustaining perishable skills, such as mine detector operation, through the application of CAI technology has proven effective. The CAI system used in this study may have led to issues that Alexander et al. (2005) refer to as buy-in. Operators may not buy-in to a CAI system that makes use of a mouse vice a replica of the model of mine detector they normally operate and buy-in may require a virtual reality concept that immerses operators into the minefield environment and requires the operator to physically negotiate a mine cell by walking through the virtual minefield detecting targets while advancing. Developing a portable arcade-type game for mine detection would provide a tool to trainers and operators to use in maintaining operator skill and readiness at a high level. The unexpected performance of the Control group validates the thoroughness of the familiarization training conducted with all participants on the first day of the trial. It also reflects positively on the F-3s manufacturer for developing a quality, easy to operate instrument. The performance of the Control group in this study suggests that to truly evaluate the effectiveness of the CAI as a sustainment tool, a period of non-mine detector use is required. We will make preparations to conduct a follow-on study with this group of participants to enable us evaluate the CAI methodology as a means to sustain mine detector operator skills. Conclusions While no significant differences were observed among the training methodologies in this study, the target detection performance of participants receiving CAI training is on par with participants practicing target detection prior to testing and the Control group. In other words, the CAI did not significantly degrade operator performance. The experimental methodology may have provided insufficient time for skill erosion to develop in the participants, and as such statistically supportable inferences based on training methodology when compared to a control group could not be made on the studys results. Trends showed practice improves average performance in terms of targets found 8.3% and 5.5%, and targets located within distance 8.6% and 6.1% above CAI and no practice, respectively (N=27). The results also suggest that men and women can effectively perform the task of mine detection. 73

Table 5-1 Differences among training methodologies when N= 27

Dependent Variable Average Distance

Training Method Training Method Significance

Control

CAI Practice

.924 .839 .981

CAI

Practice

Targets Found (log)

Control

CAI Practice

.720 .163

CAI

Practice

.035*

Targets Found Within Distance

Control

CAI Practice

.916 .409

CAI

Practice

.226

* Comparison is significantly different at the 0.05 level. Table 5-2 Differences among training methodologies when N=26

Dependent

Training Method Training Method Significance 74

Variable Average Distance CAI Control CAI Practice Practice .997 .836 .878

Targets Found (log)

Control

CAI Practice

.945 .134

CAI

Practice

.082

Targets Found Within Distance

Control

CAI Practice

1.00 .384

CAI

Practice

.408

75

Figure 5-1. Average targets found by training methodology and gender.

76

Figure 5-2. Average targets located within allowable distance by training methodology and gender.

77

Figure 5-3. Average distance of chip placement from actual target center.

78

Table 5-3 Time on Task By Training Method test lane only Training Method Control Mean 66.4 Maximum Time on Task 98 Minimum Time on Task 37

CAI

60.4

100

35

Practice

58.4

93

35

Figure 5-4. MD investigation of a target with spiral technique.

Mine body (GPR signal) Halo (MD signal) MD Alerts

9 oclock approach

3 oclock approach

6 oclock approach

79

References
Alexander, A. L., Brunye, T., Sidam, J., and Weil, S.A (2005). From gaming to training: a review of studies on fidelity, immersion, presence, and buy-in and their effects on transfer in PC based simulations and games. Retrieved on 4/10/2008 from http://www.darwars.bbn.com/downloads/DARWARS%20Paper%2012205.pdf Cyterra. (2006). Countermine. Retrieved December 1, 2006 from http://www.cyterra.com/countermine.html. Frank Rotondo, F., & Ayers, L. (2003). Check Test 7 (Oct. 2003): Comparison between Snow And No-Snow Cover using the HSTAMIDS Mine Detector. Memorandum to NVESD. Alexandria, VA: Institute for Defense Analyses, Science and Technology Division. Guckert, R. (2000). HSTAMIDS Red Team experiences/Lessons learned. Proceedings of the UXO/Countermine Forum, 2-4 May 2000, Anaheim, CA. Hancock, R.A. (2006). Decay in AN/PSS-14 operator skill at 30, 60, &90 days following training: a report to Dr. Alan Davison, Cooperative Agreement Manager, on Cooperative Research Contract W911NF-05-2-0029. Lincoln University, Jefferson City, MO. Hancock, R. A. (2006). Decay in AN/PSS-14 Operator Skill at 30, 60, & 90 Days Following Training:
A Report to Dr. Alan Davison, Cooperative Agreement Manager, on Cooperative Research Contract W911NF-05-2-002. Jefferson City, Missouri: Lincoln University, Department of

Social & Behavioral Sciences, Lane Mine Detection Research Center. Herman, H., & Inglesias, D. (1999). Human-in-the-loop issues for demining. In T. Broach, A. C. Dubey, R. E. Dugan, and J. Harvey, (Eds.), Detection and Remediation Technologies for Mines and Minelike Targets IV, Proceedings of the Society for Photo-Optical Instrumentation Engineers 13th Annual Meeting, Vol. 3710, 797-805. Herman, H., McMahill, J., & Kantor, G. (2000). Training and performance assessment of landmine detector operator using motion tracking and virtual mine lane. In A. C. Dubey & J. F. Harvey & J. T. Broach & R. E. Dugan (Eds.), Detection and remediation technologies for mines and minelike targets V, Proceedings of the Society for PhotoOptical Instrumentation Engineers 13th Annual Meeting, Vol. 4038. Orlando: SPIE.110121. Maurer, L. (2003). AN/PSS-14 Mine Detector Assessment for Operation Enduring Freedom.

80

NVESD Trip report. Mincey, J. C. (2005). Personal communication. U.S. Army Engineer School, Counter Explosive Hazards, Center, Fort Leonard Wood, MO, October 2005. Santiago, A., Locke, M., & Reidy, D. (2004). Operational test results for the AN/PSS-14 (HSTAMIDS). Proceedings of the UXO/Countermine Forum, 9-12 March 2004 St.Louis, MO. Schweitzer, K, Davis, B., Pettijhohn, B. A., Clark, R. D., Davision, A. D., and Staszewski, J.(2006). Optimization of AN/PSS-14 Operator Training. Army Research Labortary Technical Report #3970. Aberdeen Proving Ground, MD: Army Research Laboratory. Staszewski, J. (2001). Training for Operators of Handheld Detection Equipment: A Cognitive Engineering Approach. Proceedings of the UXO/Countermine Forum, 9-11 April 2001, New Orleans, LA. Staszewski, J., & Davison, A. (2002). Supporting the Army transformation through cognitive engineering: Evidence from improved landmine detection capability. 2002 Army Science Conference, Orlando, Fl, 2-6 December 2002. Stefanidis, D., Korndoffer, J., Sierra, R., Touchard, C., Dunne, J., and Scott, D. (2005). Skill retention following proficiency-based laparoscopic simulator training. Retrieved on 4/10/2008 from http://linkinghub.elsevier.com/retrieve/pii/S003966005002722 United Nations Mine Action Service. (2006). HSTAMIDS operational with deminers in Cambodia, Afghanistan, and Thailand. Mine Action Technology Newsletter, June 2006, No. 4. 8-9. Retrieved December 1, 2006 from http://www.cyterra.com/news/PSS14%20Article%20UNMAS%20Tech%20NL%20June%2006.pdf

81

Alphabetical List of Acronyms

Acronym ANOVA APLM APM ATLM ATM CAI FAR GPR MD PMCS POI RP SIM/s SMS SS

Meaning Analysis of Variance Anti-Personnel Low Metal Mine Anti-Personnel (High) Metal Mine Anti-Tank Low Metal Mine Anti-Tank (High) Metal Mine Computer Aided Instruction False Alarm Rate Ground Penetrating Radar Metal Detector Preventative Maintenance Checks and Services Program of Instruction Research Participant Simulant (Land Mine Simulant) Sweep Monitoring System Short Sweep for GPR

82