854 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO.

8, AUGUST 1996

Automating Image Processing for Scientific
Data Analysis of a Large Image Database
S t e v e A. C h i e n and H e l e n B. Mortensen
Abstract-This article describes the Multimission VICAR Planner
(MVP): an AI planning system which uses knowledge about image
processing steps and their requirements to construct executable image
processing scripts to support high-level science requests made to the
Jet Propulsion Laboratory (JPL) Multimission Image Processing
Subsystem (MIPS). This article describes a general AI planning
approach to automation and application of the approach to a specific
area of image processing for planetary science applications involving
radiometric correction, color triplet reconstruction, and mosaicing in
which the MVP system significantly reduces the amount of effort
required by image processing experts to fill a typical request.
Index Terms-Data analysis, image processing, artificial intelligence
(AI), planning, automated programming.
1 PROBLEM
DESCRIPTION
THE VICAR environment (Video Image Communication and Re-
trieval') 1131 supports image processing for: JPL flight projects
including VOYAGER, MAGELLAN, and GALILEO, and CASSINI;
other space imaging missions such as SIR-C and LANDSAT; and
numerous other applications including astronomy, earth re-
sources, land use, biomedicine, and forensics. Outside of JPL,
VICAR users include universities, the military, research institu-
tions, aerospace corporations, companies, and Home Institution
Image Processing Subsystem (HIIPS) sites with over 100 users.
VICAR allows individual processing steps (programs) to be
combined into more complex image processing scripts called pro-
cedure definition files (PDFs). As one of their primary duties, JPL
analysts construct PDFs to achieve tasks such as image correction,
image enhancement, construction of mosaics, creation of movies
and rendering of objects. Individual processing programs perform
many different data processing functions such as photometric
correction, radiometric correction, and line fillin.'
Unfortunately, manual construction of VICAR scripts is both
labor and knowledge intensive. Because of the complexity and
amount of program knowledge relevant to the task as well as the
many interacting problem goals, VICAR procedure generation is a
labor intensive task. Generation of a highly complex VICAR pro-
cedure may take u p to months of analyst time. The VICAR proce-
dure generation problem is also a knowledge intensive task in that
an analyst must possess knowledge of:
1) image processing and image processing programs (as of
1/93 there were approximately 50 frequently used pro-
grams, some having tens of options),
2) database organization and labels to understand current data
state, and
3) the VICAR programming language to produce and store
relevant information.
Generally, a VICAR user with 1-2 years of experience is consid-
ered a novice VICAR user, while it may take 4-5 years to become a
VICAR expert.
In the remainder of this article, we first give an example of
MVP usage. Next, we describe the basic architecture of the MVP
system. and the novel features of the MVP system from the per-
spective of planning technology. This is followed by a description
of the significance and impact of the specific VICAR application as
well as a discussion of the generality of the approach. Finally, we
describe related work and conclusions.
2 AN EXAMPLE
OF MVP USAGE
In order to illustrate how MVP assists in VICAR planetary image
processing, we now provide a typical example of MVP usage to
ground the problem and the inputs and outputs required by MVP.
The three images, shown at the left of Fig. 1 are of the planet Earth
taken during the Galileo Earth 2 flyby in December 1992. How-
ever, many corrections and processing steps must be applied be-
fore the images can be used. First, errors in the compression and
transmission of the data from the Galileo spacecraft to receivers on
Earth has resulted in missing and noisy lines in the images. Line
fillin and spike removal are therefore desirable. Second, the im-
ages should be map projected to correct for the spatial distortion
that occurs when a spherical body is represented on a flat surface.
Third, in order to combine the images, we need to compute com-
mon points between the images and overlay them appropriately.
Fourth, because we are combining multiple images taken with
different camera states, the images should be radiometrically cor-
rected before combination.

1. This name is somewhat misleading as VICAR is used to process

considerable nonvideo image data such as MAGELLAN synthetic
aperture radar (SAR) data.
2. For example, the PHOTFUNC program can photometrically cor-
rect an image for lighting conditions due to the relative position of the Fig. 1. Raw and processed image files.
lighting source, imaging device, and target and the FICOR77 program
can radiometrically correct VOYAGER images for varying camera MVP enables the user to input image processing goals through
response depending on camera state and other properties.
a graphical user interface with most goals as toggle buttons on the
interface. A few options require entering some text, usually func-
* The authors are with thelet Propulsion Laboratory, California Institute of tion parameters that will be included as literals in the appropriate
Technology, 4800 Oak Grove Drive,Pasadena, C A 91 109-8099. place in the generated VICAR script. Fig. 2 shows the processing
E-mail: {steue.chien,helen.morteizsen}~jpl.nasa.gou.
goals input to MVP. Using the image processing goals and its
Manuscript veceiued Mar. 14,1995. Recommended for acceptance by K . Bowyer knowledge of image processing procedures, MVP constructs a
For information on obtaining reprints of this article, please send e-mail to:
plan of image processing steps to achieve the requested goal. This
transpnmi~computeu.or~, and reference I E E E C S Log Number P96063.

0162-8828/96505.00 01996 IEEE

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 23, 2009 at 21:39 from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8, AUGUST 1996 855

plan is translated into a VICAR script which, when run, performs has properly satisfied the request. In most cases, upon inspection,
the desired image corrections and constructs a mosaicked image of the analyst determines that some parameters need to be modified
the three input files. The finished result of the image processing subjectively or goals reconsidered in context. This process typi-
task is shown in Fig. 1 on the righLt. The three original images now cally continues several iterations until the analyst is satisfied with
appear as a single mosaicked image, map projected with missing the image product.
and corrupted lines filled in.
3 THEMULTIMISSION
VICAR PLANNER
(MVP)
radiometric correction pixel spike removal
MVP [31, [4] partially automates generation of image processing
missing line fillin uneven bit weight correction
procedures from user requests and a knowledge-based model of
no limbs present in images perform automatic navigation
perform manual navigation an image processing area using Artificial Intelligence (AI) auto-
display automatic nav residual error
display man nav residual error map project with parameters... mated planning techniques [9], [15], [17]. In AI planning, a system
mosaic images smooth mosaic seams using DN uses:
1) a model of available actions in a domain;
Fig. 2. Example problem goals. 2) a model of the current state; and
3) a goal specification;
To further continue this example, shown in Fig. 3 is a code to determine the actions necessary to achieve specified goals. In
fragment to perform portions of image navigation3 for a Galileo VICAR image processing, the actions are VICAR image processing
image.4 The higher-level conceptual steps (i.e., plan steps) are programs, the current state is the current state of the image files of
shown at the left and the corresponding VICAR code is shown at interest, and the specification of the desired state corresponds to
the right. In this case the overall user goal is to navigate the image. the user image processing goals. By partially automating the filling
The other subgoals (and steps) are necessary to support this goal. of basic science image processing requests: Image processing re-
Conceptual Stem VICAR Code quest turnaround time is reduced, analyst time is freed for more
complex and challenging science requests, and analyst workload is
- IBISNAV OUT="file-list N A V f~LANET=targelLO_lO +
get initial navigation
information
PROJECT="GLL SEDR=@iRIMSRC
" FILENAME="file~list
lllstl reduced. Additionally, encoding image processing knowledge in
) I Construct initial overlap
pairs MOSPLOT MVP allows this knowledge to be retained by institutions, rather
MOSPLOT Inp="flle_llst NAV" inl=llnes%O_Sns=samples_0-6 propct="GLL
COnStlUC~ lnltial
overiap pairs - I mos overlap IS just a holder for the overlap plot
"
than being lost when analysts leave or retire.
dcl copy prinlionx plt mo5 overlap
dcl pnntlndeed mon overlap From a technology standpoint, MVP is significant in several re-
reiine initial - ( 1 Refine initial overlap pairs edibis spects. First, MVP integrates multiple planning paradigms to most
pairs EDIBIS INP="file_list OVER"
naturally represent domain constraints and human experts' prob-
lem-solving methods. Second, MVP uses, novel methods to repre-
sent and reason about VICAR program option constraints. Third,
11 The following code IS in wrllltin VMS
tt LOCAL STR STRING INlT = " " MVP embodies an AI planning approach to solving complex pro-
cedural automation problems such as automated image process-
tiepoint file DCL DEASSIGN NAME
(if present) DCL DEFINE NAME 'F$SEARCH('flle_list T P ) ing. We believe our approach is general ,and extends to other pro-
LOCAL STR STRING
TRANSLOG NAME STR cedure automation tasks-we later describe other applications of
LET _ONFAIL = "RETURN" 11 !Set PDF to return on error
the MVP engine to other procedure autiomation tasks to support

use manmatch
/ IF (STR = "")
MANMATCH INP=(7lle~llslNAV~YiIe_llsIOVER) +
oUT="file_l~stT P PROJECT="GLL " 'SEDR FILENAME="flle_l!st ILIST
this claim. From an applications standpoint, MVP is significant in
that MVP is a successfully deployed AI planning application
p r o g r a m to I1 If an old liepoint file exists
COlStrUCt Or
refine
file \ 41 . IS .art of i n.w t and lateiovewrltten
The old tDfile

+
E%NMATCH INP=('file_list NAV,"file_list.OVER."f~le_l~sl.TP')
OUT="t~le_l~st
T P PROJECT="GLL " 'SEDR FILENAME="file_lisl iLlST
which has had considerable impact in a specific VICAR planetary
imaging task-we later describe the impact of the MVP system.

1 1 OMCORP 3.1 The MVP Architecture

~
OM
~ ~ ~ ~ $ $ p
matrix
OMCORP INP=rflle_llst NAV","file~l~st TP") PROJECT="GLL " GROUNO=BGOOD
OMCORP I N P = ~ f i l e hNt A V , " f i l e h t T P ) PROJECT="GLL " GROUND=BGOOD
The overall architecture for the MVP system is shown in Fig. 4.
MVP uses two planning paradigms: decompositional planning [61,
Fig. 3. Sample VICAR code fragment.
[llI5and operator-based planning [15]. Because these planning
approaches are well understood in the planning literature, we
Thus MVP allows the user to go directly from high level image
focus on the adaptation of these methods in MVP to the image
processing goals to an executable image processing program. By processing domain.
insulating the user from many of the details of image processing, To use MVP, the user inputs a problem specification consisting
productivity is enhanced. The user can consider more directly the of processing goals and image information using a menu-based
processing goals relevant to the end science analysis of the image,
graphical user interface. These goals and image information are
rather than being bogged down in details such as file format, nor-
then passed to the decomposition-based planner. The decomposi-
malizing images, etc.
tion planner uses decomposition rules to irnplement two conceptual
MVP does not always fully automate this planetary imaging
types of planning. First, the decomposition-based planner uses
task. In typical usage, the analyst receives a request, determines
image processing knowledge to classify the overall problem type
which goals are required to fill the request, and runs MVP to gen-
which the user has specified in a process 'called skeletal planning [91.
erate a VICAR script. The analyst then runs this script and then
Second, the decomposition planner uses this classification to de-
visually inspects the produced irnage(s) to verify that the script
compose the problem into smaller subproblems in a process called
hierarchical planning 1171.
3. Image navigation is the process of determining the matrix trans- The subproblems produced by the decomposition planner are
formation to map from the two-dimensional (line, sample) coordinate then solved by operator-based planning [151, in which a planner uses
space of an image to a three-dimensional coordinate space using in- a description of possible actions (in this case image processing
formation on the relative position of the imaging device (spacecraft
position) and a model of the target being imaged (e.g., the planetary
body). 5. This approach has also been called Hierarchical Task Network
4. This code was generated by MVP. Planning or Task Reduction planning.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 23, 2009 at 21:39 from IEEE Xplore. Restrictions apply.
 856 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8, AUGUST 1996

cause skeletal and hierarchical planning are not unique to MVP, we

only give examples of how they are used in MVP. For a more de-
tailed treatment of these concepts the reader is referred to [9], [171.
Skeletal planning [91 is an approach to planning in which a
planner classifies a problem using the goals and initial state and
uses this classification as a guide to solving the problem. MVP
uses decomposition rules to determine the general problem class
and uses this problem class to decompose the problem into sub-
problems. For example, Fig. 5 shows a decomposition rule for the
problem class mosaicking with absolute navigation. This rule states
that if mosaicking is a goal of the problem and an initial problem
decomposition has not yet been made, then the initial problem
decomposition should be into the subproblems local correction,
navigation, registration, mosaicing, and touch-ups and that these
Fig. 4. MVP system architecture subproblems must be solved in that order.

steps) to solve subproblem goals as indicated by the problem de- LHS RHS
composition. The resulting plan segments are then assembled us- GI= mosaicking goal present GR = 1. local
ing constraints derived in the decomposition process. The result- correction,
ing plan is then used to generate an actual executable VICAR PDF CO= null 2. navigation
using conventional macro expansion techniques. C2= an initial classification 3. registration
From an AI planning technology standpoint, MVP uses both has not yet been made 4. mosaicking
decomposition and operator-based planning techniques. MVP 5. touch-ups
uses both techniques for two reasons: search control and user under- c, = these subtasks be
standability. The decomposition approach is needed for search performed in order
control. Plans in the MVP domain can be of considerable length (up 1. 2. 3. 4. 5.
protect local correction
to 100 steps) and each step (or VICAR program) can involve rea- until mosaicking
soning about numerous complex effects (often operators have tens of N= problem class is
effects).' Due to the large search space caused by this complexity, mosaicking
conventional operator-based planning approaches are not able to
tractably construct plans in the VICAR domain without significant
Fig. 5. Skeletal planning decomposition rule.
control knowledge. In the decomposition planning paradigm, it is
natural to encode knowledge on how to break u p a large problem
into smaller subproblems. In the decomposition component, MVP In hierarchical planning [17], abstract goals or procedures are
breaks up a large search space planning problem caused by the incrementally refined into more and more specific goals or proce-
complexity of the image processing tasks into several smaller prob- dures in context. MVP uses hierarchical decomposition to refine
lems, thus reducing the search encountered during operator-based the initial skeletal plan into a specialized plan based on the specific
planning. Indeed, the problem decomposition rules used in MVP current goals and situation. This allows the overall problem de-
can be considered a very important form of search control knowl- composition to be influenced by factors such as the presence or
edge essential to MVP's image processing capability. absence of certain image calibration files or the type of instrument
MVP also uses decomposition-based planning for reasons of and spacecraft used to record the image. For example, the decom-
user understandability. MIPL analysts do not consider an entire position rule in Fig. 6 states that if the limb is present in all of the
image processing problem all at once. Typically, analysts begin by images7 for VOYAGER and GALILEO images, the navigation step
classifying the current problem and using this classification to can be performed by absolute navigation (a process in which each
decompose the problem into smaller subproblems. Because MVP of the images can be navigated independently).
uses decomposition-based planning to perform a similar process,
LHS RHS
MVP can produce an annotated trace of how the overall problem
GI= navigation action present GR = 1. absolute
was classified and decomposed. This annotated trace greatly as-
CO= null navigation
sists the analyst user in understanding MVP output.
C2= the project is VOYAGER C1 = null
3.2 Skeletal and Hierarchical Planning in MVP or GALILEO and N = null
MVP first uses the decomposition planning framework to break an limb present in all images
image processing problem into smaller subproblems-then solves
the resulting subproblems using operator-based planning tech- Fig. 6. Hierarchical refinement decomposition rule.
niques. MVP breaks a problem into subproblems using decomposi-
tion rules to perform skeletal planning and hierarchical planning. Be- Using decomposition methods for skeletal and hierarchical
planning in MVP has several strengths. First, the decomposition
6. However, it is worth noting that a VICAR script for a specific re- rules very naturally represent the manner in which the analysts
quest generally does not contain complex control constructs (e.g.,con- attack the procedure generation problem-facilitating knowledge
ditionals, looping). Most VICAR scripts to fill a single request would engineering. Second, the notes from the decomposition rules used
contain few (if any) conditionals and these few conditionals are easily to decompose the problem can be used to annotate the resulting
handled in the macro expansion phase. Most VICAR scripts for a spe- PDF to make the VICAR programs more understandable to the
cific request also do not contain loops. In the case where loops occur, analysts. Third, relatively few problem decomposition rules are
they are generally looping over a finite set (such as over a known set of
image files). Thus the more general, difficult problems of automated
programming such as determining loop invariants and termination 7. This means that the sunward edge of the planet is visible in all of
criteria are not relevant to the VICAR application domain. the images.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 23, 2009 at 21:39 from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8, AUGUST 1996
easily able to cover a wide range of problems and decompose
them into much smaller subproblems.
3.3 Operator-Based Planning in MVP
MVP uses classical operator-based planning techniques to solve
subproblems produced by the decomposition-based planner D51.
An operator-based planner uses models of actions in a domain to
achieve goals from an initial world state. In the VICAR domain the
actions (operators) are image processing steps, initial state the
initial image file state, and the goals the processing request.
In operator-based planning, an action is represented in terms of
its preconditions (required to be true before an action can be exe-
cuted), and its effects (true after an action is executed). For exam-
ple, the GALSOS program to radiometrically correct Galileo image
files is represented as shown below.
operator GALSOS
:parameters ?infile ?utbwc ?calc
:preconditions
the project of ?infile must be galileo
the data in ?infile must be raw data values
:effectsreseaus are not. intact for ?infile
the data in ?infile is not raw data values
missing lines are not filled in for ?infile
?infile is radiometrically corrected
the image format for ?infile is halfword
?infile has blemishes-removed
if (UBWC option is selected)
then ?infile is uneven bit weight corrected
if (CALC option is selected)
then ?infile has entropy values calculated
When constructing a plan to achieve a goal G1, a planner will con-
sider those actions which have G1 as an effect (thus considering
GALSOS to achieve a radiometric correction goal). In order to use
GALSOS, MVP must also ensure that the preconditions of the op-
erator are met, in a process callecl subgodling. MVP must also en-
sure that operators in the plan do not undo conditions required by
other parts of the plan-this is performed in a process called con-
flict
One novel aspect of the VICAR domain is that considerable
search in planning is not at the program selection level (which corre-
sponds to operator selection in the planning process) but rather at
the pvogvum option selection level (which corresponds to selecting the
appropriate operator effect after the operator has been selected).
Thus, when planning to achieve a goal, MVP searches more to
determine how to set program options to achieve a goal rather
than to determine which VICAR program (planning operator) to
use to achieve the goal. This presents a problem for efficiently
reasoning about interacting program options (operator effects) in
that certain combinations of program options (operator effects) are
inconsistent (i.e., cannot be used together).
For example, when considering operators to achieve a perspec-
tive correction, MVP might need to consider a family of map pro-
jection programs (MAP, MAP3, and MAP41 and the rotational
correction program PTP. This search to find the correct program is
not overly difficult. However, for a particular problem, after having
selected PTP, MVP would need to determine which method to use to
specify the spacecraft pointing information. There are several ways
in which the pointing information could be computed. MVP could
use existing navigation SEDRSRC data, MVP could directly specify
target information using the PC and RPC options, or MVP could
regenerate navigation data from basic NAIF using one of many
8. Because subgoaling and conflict analysis in operator-based plan-
ning are not unique to MVP, we have only briefly sketched their key
elements. For a more detailed treatment of operator-based planning
algorithms the reader is referred to 1151.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 23, 2009 at 21:39 from IEEE Xplore. Restrictions apply.
857
methods including NEARENC, FARENC, MANMATCH, AUTO-
MATCH. In using the PTP program, program options are set to indi-
cate the method being used. Each of these methods has its own re-
quirements which are represented as a set of program options speci-
fications and preconditions for a Conditional effect of the operator.
Below is the conditional effect corresponding to using the SEDRSRC
pointing specification to retrieve FARENC data for the PTP program
in which the SEDRSRC specification is incompatible with the PC and
RPC options (" indicates a program option constraint).
IF
(the SEDRSRC pointing specification is used
and SEDRSRC is specified to be FARENC)* and
(the PC and RPC pointing specification is not
used)* and
the project of file is V0YAGER:L or VOYAGER2 and
appropriate SEDR data files for file exist and
the camera number RCAM for the file has been
correctly specified and
the FDS for file has been correctly specified
THEN
the output image outfile will be registered to the
reference image as speci:Eied
When using an operator effect to achieve a subgoal in the plan,
MVP first checks to see if the program option preconditions are
consistent with the plan, only then allowing the effect to be used
(and adding the program option constraints to the plan).' This
method for representing VICAR program options is important in
that it allows for independent program options to be reasoned
about and constrained independently yet represents the interac-
tion between conflicting options. For example, using this
SEDRSRC option affects possible PC and RPC option usage, but it
does not affect the usage of other PTP program options such as
ROT, RE, and RP.
This ability to directly to specify interacting options is a unique
capability of the MVP planner and represents a capability that
other operator-based planners do not have. Consequently, in order
to represent incompatible program options other planners would
need to either:
1) use contradictory preconditions to enforce disallowed com-
binations; or
2) break inconsistent operator effects, into different planning
operators (with each operator representing a consistent
combination of operator effects).
Option 1 would require the ability to easily detect inconsistent
preconditions when choosing an effect and is analogous to our
method (but more complicated). Not detecting these contradictory
preconditions when choosing an effect would cause considerable
unnecessary search. Option 2 (breaking inconsistent effects
(program options) into separate operator:;) requires an increase in
the number of operators exponential in the number of inconsistent
options (N pairs of incompatible options requires 2N operators).
Even worse, when selecting an operator which one option de-
cided, the planner would have to arbitrarily commit to decisions
on other program options-potentially causing unnecessary
search. As the number of program options can be quite large
(frequently in the tens of options), these are important representa-
tional and search efficiency issues.
9. MVP allows positive and negative program options constraints
(i.e., if option A is used then option B may not be used; and if option C
is used option D must be used). These types of constraints can be sup-
ported analogously to codesignation const:raints using the UNION-
FIND algorithm in polynomial time.
 858 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8, AUGUST 1996
3.4 On the Impact of Combining Decomposition and
Operator-based Planning Methods
Earlier we stated that the two reasons for combining decomposi-
tion and operator-based methods were user understandability and
search control. While it is difficult to quantify increased under-
standability of plans, in this section we attempt to roughly quan-
tify the effectiveness of decomposition methods in controlling the
search required by the operator-based planner.
The principle impact of decomposition planning on search is to
decompose the planning process into independent subproblems
which can be solved independently in a known sequential fashion.
In the current MVP knowledge base, there are seven such problem
spaces-Fig. 7 describes the number of relevant planning opera-
tors, the number of top-level input goals, and the typical number
of plans searched in the problem space. The overall effect of de-
composition planning on search is to break down the search into
more manageable subproblems. For example, if subproblem A
typically requires searching a plans and subproblem B typically
requires searching p plan, solving both problems simultaneously
(using uninformed search such as breadth-first) would require on
order a! p plans." Overall, because the search spaces combine
(roughly!) multiplicatively, the impact of adding domain knowl-
edge to decompose subproblems has been enormous. For example,
originally the automatic navigation and manual navigation prob-
lem spaces were represented as a single navigation problem space.
However, this problem space required too much search (on the
order of 50,000 plans), so it was broken into the automatic and
manual navigation problem spaces.
Problem Space # of operators goals typ. search
local correction 15 7 60
automatic navigation 20 4 150
manual navigation 24 4 300
photometric correction 5 2 60
registration 13 5 110
mosaicing 4 3 325
touch ups 10 3 325
Fig. 7. Problem space information
4 DISCUSSION,
RELATEDWORK AND CONCLUSIONS
MVP is significant in that it is a successfdy deployed solution to
planetary image processing for radiometric correction, color triplet
reconstruction, mosaicing with relative or absolute navigation, regis-
tration, and simple filtering and stretching tasks. MVP supports
roughly 70 VICAR subroutines and the MVP knowledge base in-
cludes about 50 operators, 50 decomposition rules, and tracks roughly
70 attributes per image file. The produced plan for a complete prob-
lem may contain over 100 operators, with a typical plan containing 60
planning operators and corresponding to a 100 line VICAR script.
In order to assess the impact of the MVP system, we asked
analysts to estimate the effort required to satisfy typical requests
for the problem areas listed above. The analysts estimated that for
these tasks MVP reduces effort to generate an initial PDF for an
expert analyst from 4 hours to 15 minutes and reduces the effort
for a novice analyst from several days to 1 hour. The analysts also
judged that the quality of the PDFs produced using MVP are com-
parable to the quality of completely manually derived PDFs. While
these results certainly do not represent a rigorous controlled em-
pirical study (which would be difficult due to the variability of
requests and the scarcity of analyst time), these results do repre-
sent strong evidence as to the usefulness of the MVP system.
10. Depending on the impact of dividing problems on heuristics it
could be either worse or better. Empirically in the MVP image proc-
essing application combining two search spaces A and B as above
would result in search slightly less than a!P.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 23, 2009 at 21:39 from IEEE Xplore. Restrictions apply.
From another perspective, the MVP system represents a gen-
eral approach of using AI Planning Technology as a solution to
procedure automation problems. On this front, the MVP planning
engine has been adapted for generating procedures to operate JPL
Deep Space Network Antennas to communicate with spacecraft
[81. This system, called DPLAN, was demonstrated in February
1995 and is currently being deployed at Deep Space Network
(DSN) complexes. Work is also underway to field another version
of MVP for a planetary geology group at the department of Geol-
ogy at Arizona State University to perform synthetic aperture ra-
dar (SARI image processing. Our success in using AI planning
representations in the MVP at JPL, DSN application, and planetary
geology applications represent evidence of the generality of the
approach.
Related work can be broadly classified into: related image
processing languages, related automated image processing work,
and related AI planning work. Many commercial and academic
image processing packages (e.g., IDL, Aoips, and Merlyn), have
only limited predefined ability to automatically determine how to
use different image processing programs or algorithms based on
the problem context. However, there are several previous systems
for automatic image processing that use a domain independent
mechanism. Charlebois et aI. [Z] describe a case-based system for
image processing and acquisition of image processing knowledge
while MVP uses decompositions and an operator-based approach.
Other systems do not have a strict semantics on the actions in the
domain and hence do not guarantee correct handling of negative
goal interactions: Jiang and Bunke [lo], Matsuyama [141, Grirnm
and Bunke [7], and Sakaue and Tamura [16]. In contrast, AI plan-
ning techniques have a strict semantics 161, [15] and guarantee
correct handling of goal interactions. Other work by Zmuda et al.
[181 automatically derives classification software by using machine
learning techniques. However, for the MVP applications, the
search space of possible programs is too large and there is no end
feedback (as in classification) to drive the learning process. The
SAT1 system 111 uses an interactive dialogue with the user to drive
an automated programming approach to generating code to sat-
isfy the user request. OCAPI [5], a semantically integrated auto-
mated image processing system, while being very general pro-
vides no clear way to represent the large number of logical con-
straints associated with the problems MVP was designed to solve.
The COLLAGE [12] image processing system uses solely the de-
composition approach to planning and differs from MVP applica-
tion-wise in that it focuses primarily on earth imaging applications
in the Khoros environment, where MVP has focused on planetary
applications in the VICAR environment.
This paper has described the application of AI planning tech-
niques to automate image processing by the MVP system. This
work is a significant advance in the state of the art in AI planning
technology in that:
1) it represents an integration of decomposition and operator-
based planning paradigms; and
2) it uses explicit constraints to efficiently reason about opera-
tor effects.
The AI planning approach represents a general approach to auto-
mating procedure generation problems and we presented evi-
dence to support this view. The work described in this paper is
also significant from an image processing applications perspective
in that MVP reduces the effort to perform the specific VICAR im-
age processing tasks of: radiometric correction, color triplet recon-
struction, and mosaicing for experts from four hours without MVP
to 15 minutes with MVP.
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOI
ACKNOWLEDGMENTS
This work was performed by the Jet Propulsion Laboratory, Cali-
fornia Institute of Technology, under contract with the National
Aeronautics and Space Administration. The MVP task was funded
in part by the Operations/Artificial Intelligence Program, Code X,
managed by Melvin Montemerlo and in part by the Continuous
Improvement Initiative, Telecommunications, and Mission Opera-
tions Directorate of JPL managed by Barbara Anderson. Other past
and present members of the MVP team are Christine Ying, Shouyi
Hsiao, Forest Fisher, Darren Mutz, Alex Gray, Joe Nieten, and Jean
Lorre.
REFERENCES
0. Capdevielle and P. Dalle, ”Image Processing Chain Construc-
tion by Interactive Goal Specification,” Proc. First IEEE Int’l Conf.
Image Processing, vol. 3, pp. 816-819, Austin, Tex., Nov. 1994.
D. Charlebois, J. DeGuise, G. Goodenough, S. Matwin, and M.
Robson, ”A Case-Based Planner to Automate Reuse of ES Soft-
ware for Analysis of Remote Sensing Data,” Proc. Int’l Geoscience
and Remote Sensing Symp. (IGARSSI, vol. 3, pp. 1,851-1,854, 1991.
S. Chien, ”Automated Synthesis of Image Processing Procedures
for a Large-scale Image Database,” Proc. First IEEE Int’l Conf. Im-
age Puocessing, vol. 3, pp. 796-800, Austin, Tex., Nov. 1994.
S. Chien and H. Mortensen, “Automating Image Processing for
Scientific Data Analysis of a Large Image Database: Extended Re-
port,” JPL Technical Document D-13463, Jet Propulsion Labora-
tory, Pasadena, Calif., Mar. 1996.
V. Clement and M. Thonnat, “A Knowledge-based Approach to
Integration of Image Processing Procedures,” Image Understand-
ing, vol. 57, pp. 166-184, Mar. 1993.
K. Erol, J. Hendler, and D. Nau, ”UMCP: A Sound and Complete
Procedure for Hierarchical Task Network Planning,” Proc. Second
Int’l Conf. AI Planning Systems, pp. 249-254, Chicago, June 1994.
F. Grimm and H. Bunke, ”An Expert System for the Selection and
Application of Image Processing Subroutines,” Expert Systems,
vol. 10, no. 2, pp. 61-74, May 1993.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 23, 2009 at 21:39 from IEEE Xplore. Restrictions apply.
18, NO. 8, AUGUST 1996 859
R. Hill, S.Chien, K. Fayyad, C. Smyth, T. Santos, R. Chen, and R.
Bevan, ”Sequence of Events Driven Automation of The Deep
Space Network,” Telecommunications and Data Acquisition, Winter
1995.
Y. Iwasaki and P. Friedland, ”The Concept and Implementation of
Skeletal Plans,” 1. Automated Reasoning, vol. 1, no. 1, pp. 161-208,
1985.
X. Jian and H. Bunke, “Vision Planner for an Intelligence Mul-
tisensory Vision System,” Technical Report, Univ. of Bern
(extended version of a paper appearing in ICPR 1994).
A. Lansky, “Localized Planning with Diverse Plan Construction
Methods,” TR FIA-93-17, NASA Anies Research Center, June
1993.
A. Lansky, M. Friedman, L. Getoor, S. :Schmidler,and N. Short Jr.,
”The Collage/Khoros Link: Planning for Image Processing
Tasks,” Proc. 1995 A A A 1 Spring Symp. Integrated Planning Applica-
tions, pp. 67-76, Mar. 1995.
S. LaVoie, D. Alexander, C. Avis, H. Mortensen, C. Stanley, and L.
Wainio, ”VICAR User’s Guide, Version 2,” JPL Internal Docu-
ment D-4186, Jet Propulsion Laboratory, California Inst. of Tech-
nology, Pasadena, Calif., 1989.
T. Matsuyama, ”Expert Systems for Image Processing: Knowl-
edge-Based Composition of Image Analysis Processes,” Computer
Vision, Graphics, & Image Processing, vol. 48, pp. 22-49,1989.
J.S. Pemberthy and D.S. Weld, “UCPOP: A Sound Complete,
Partial Order Planner for ADL,” Proc. Third Int’l Conf. Knowledge
Representation and Reasoning, Oct. 1992.
K. Sakaue and H. Tamura, ”Automatic Generation of Image Proc-
essing Programs by Knowledge-Based Verification,” Proc. IEEE
Conf. Computer Vision and Pattern Recog,ulition,pp. 189-192,1985.
M. Stefik, ”Planning with Constraints (MOLGEN: Part l),” Artifi-
cial Intelligence, vol. 16, no. 2, pp. 111-140, 1981.
M. Zmuda, M. Rizki, and L. Tamburino, ”Approaches to Synthe-
sizing Image Processing Programs,” Puoc. IEEE National Aerospace
and Electronics Conf., vol. 3, pp. 1,054-1,059,1991.