The Journal of Systems and Software 60 (2002) 249–261

www.elsevier.com/locate/jss

Introduction to an integrated methodology for development and

implementation of enterprise information systems
a,* b
Choon Seong Leem , Sangkyun Kim
a
Department of Computer and Industrial Engineering, Yonsei University, 134, Schincon-Dong, Seodaemoon-Ku, Seoul 129-749, South Korea
b
Security Consulting Group, KCC Information & Communication, 134, Schincon-Dong, Seodaemoon-Ku, Seoul 129-749, South Korea
Received 1 June 2000; received in revised form 1 December 2000; accepted 1 March 2001

Abstract
An integrated methodology for successful development and implementation of enterprise information systems is developed. This
paper describes the methodology and defines five components and one repository which can be customized with business scenarios
and patterns according to various business environments. Five components consist of information strategy planning, economic
justification and measurement, enterprise information system appraisal, package software evaluation, and unified modeling tools.
They characterize the methodology through its entire road map. Also, case studies are provided to prove its practical values.
2002
Elsevier Science Inc. All rights reserved.

Keywords: Integrated methodology; Enterprise information systems; Patterns; Scenario; Roadmap; Components; ISPM; EIII; EJMS; S3IE; UMT;
Repository

1. Introduction Most research works and commercial products, how-

Information technology is the important weapon to
improve and keep an enterprises’ competitiveness in
ever-changing business environment. It is a systematic
methodology that is mostly required as a supporting
tool achieving complicated activities connected with
introduction of information systems. The information
systems embodied to be impertinent can be wasting
enterprise resources and weakening enterprise’s com-
petitiveness (Jeong, 1997). Therefore, many consulting
corporations have developed and applied various com-
mercial methodologies in order to offer systematic guide
on a construction of enterprise information systems.
Methodology integrates each kinds of theory and
tools scattered and must support that all of the users
may utilize it easily (Brannback, 1993). Thus, related
methodology research must connect each kind of theory
and tools in synthetic viewpoint to satisfy efficient con-
struction efficiency and effectiveness of information
systems (Lyytinen, 1990). Also, previous researches
show that enterprises which have systematic methodol-
ogy construct more effectively information systems
(Earl, 1993).
ever, are lack of the architectural integrity and functional
applicability to meet these sophisticated needs of enter-
prises (Choi, 1998). Lack of the architectural integrity is
caused by two factors: absence of customizable architec-
ture regarding inner environment and natural culture of
enterprises, and non-integrated framework to manage
engineering tools and output data used and generated
during development and implementation of information
systems. Lack of the functional applicability is caused by
three factors: broken bridge linking business strategy with
information strategy in rational manner, absence of eco-
nomic justification and management systems, and unre-
liable mechanism for analysis and evaluation about level
of enterprise information systems.
This paper introduces a new methodology framework
to assure architectural integrity and functional applica-
bility for successful development and implementation of
information systems. A case study is also presented to
prove its practical values.
2. Previous researches
2.1. Previous works from academia

*
Corresponding author. Tel.: +82-2-2123-4011; fax: +82-2-364-7807. Related researches from academic point of view can
E-mail address: leem@yonsei.ac.kr (C.S. Leem). be classified into three areas: evaluation of information
0164-1212/02/$ - see front matter
2002 Elsevier Science Inc. All rights reserved.
PII: S 0 1 6 4 - 1 2 1 2 ( 0 1 ) 0 0 0 9 6 - 6
250 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261
systems (IS), IS investment evaluation and evaluation of
information strategy planning (ISP).
2.1.1. IS evaluation
IS evaluation usually focuses on IS success cases
which explain operational influences such as produc-
tivity, user satisfaction, profitability, etc. and models
which measure the strategic influences such as buyer and
consumers, competitive rivalry, suppliers, etc. The origin
research of IS evaluation is Delone and McLean (1992).
Table 1 summarizes some works of IS evaluation.
Mahmood (1991) made a general model to measure
potential influences of IS on an enterprise and validated
it by interviewing strategic managers and reviewing re-
searches. Palvia (1997) made a GLIT model by extend-
ing that of Mahmoods assuming conditions of an
enterprise.
2.1.2. IS investment evaluation
Theo and Berghout (1997) discerned four basic ap-
proaches such as financial approach, multi-criteria ap-
proach, ratio approach, and portfolio approach and
group evaluation approach into four classifications:
economic appraisal techniques, strategic approaches,
analytical appraisal techniques, and integrated ap-
proaches. Economic appraisal techniques are structured
in nature, and include those traditionally used by ac-
countants. They are based on the assignment of cash
values to tangible costs and benefits but largely ignore
intangible factors. Strategic approaches are less struc-
tured in nature but combine tangible and intangible
factors. Analytical appraisal techniques are highly
structured in design but subjective in nature, with their
use often including tangible and intangible factors. Fi-
nally, integrated approaches combine subjectivity with a
formal structure. These approaches integrate financial
and non-financial dimensions together, through the ac-
knowledgement and the assignment of weighting factors
to the intangible factors.
Bacon (1992) found that the criteria such as the
support of explicit business objectives and response to
Table 1
Research work on IS evaluation
Delone and
McLean (1992)
Research method Literature review
IS evaluation framework System quality
construction
Information quality
Information use
User satisfaction
Individual impact
Organizational impact
competitive systems are important in IS investment de-
cision-makings.
2.1.3. ISP evaluation
There have been relatively fewer researches on the
evaluation of ISP. King (1988) proposed an evaluation
framework addressing the entire planning process of
ISP, using input-process-output model. In another re-
search of his, the quality of ISP is related to financial
performance. Premkumar and King (1994) validated the
Kings framework by the survey analysis. Henderson and
Sinfonis (1988) emphasized the internal consistency and
the external validity in their evaluation framework.
Some evaluation framework are focused on align-
ment of business strategy with information strategy.
Reichs (1996) research developed the measure of the
alignment of business strategy with information strategy
and found significant positive relationship between the
alignment level with enterprise performance through
survey.
Sergas and Grover (1998) developed the measure-
ment model of the ISP success and its constructs are
termed as alignment, analysis, cooperation, and im-
provement in capability. Kim (1999) proposed evalua-
tion framework composed of 17 domains and it was
validated by a case study. Table 2 shows some re-
searches related to ISP evaluation.
The evaluation of ISP gained considerable interests
gradually. However, previous evaluation frameworks of
ISP failed to propose evaluation criteria at work, con-
sequently in applying them to real cases.
2.2. Previous works from industry
There are many methodologies developed by IS
solution and consulting companies. METHOD/1,
IE-Expert, and ASAP are dominant examples among
them.
METHOD/1 simplifies the process of developing
systems by breaking down each phase into smaller and
more manageable units called segments, and tasks. A
Saarinen and Scheer (1994) Grover et al. (1996) Torkzadeh and
Doll (1999)
Literature review and Literature review Literature review and
statistical verification statistical verification
Development process Infusion measure Task productivity
Use process Market measure Task innovation
Quality of the IS product Economic measure Customer satisfaction
Impact of the IS on the Usage measure Management control
organization
Perceptual measure
Productivity measure
 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261 251

Table 2
Research works on ISP evaluation
Field
Development of evaluation
framework
Development of the evaluation
framework and criteria
Evaluation focused on the alignment
of business strategy with information
strategy
Works
King (1988)
Premkumar and King (1994)
Henderson and Sinfonis (1988)
Sergas and Grover (1998)
Kim et al. (1998)
Teo and King (1996)
Reich (1996)
Contents
To address the entire planning process of ISP, using
input-process-output model
To validate the Kings framework by a survey analysis
To emphasize the internal consistency and the external
validity as evaluation dimension
To develop the measurement model of ISP success
and constructs of it are termed alignment, analysis,
cooperation, and improvement in capabilities
To propose the framework and evaluation criteria and
validate it by a case study
To find the positive relation of the alignment of business
strategy with information strategy using the stage model
To measure the alignment using two dimension,
intellectual and social linkage

series of manuals describes each of these units in detail
(Martin et al., 1997; Monheit, 1990).
The information engineering methodology (IEM) is
for planning, re-engineering and developing integrated
information systems. It provides strategies and archi-
tectures that build the basis for integrated systems.
Flexibility, which enables quick response to business
demand, is built into the methodology in the form of
multiple, alternative development paths. But, it does not
support a functionality for decision making about de-
velopment paths. Rapid application development, im-
portant for quick-strike opportunities, is integrated into
IEM. Also, the IEM includes the design of a technical
architecture, in which network systems play an impor-
tant role, while IEM does not supply functionalities
related to economic and system evaluation.
ASAP is SAP’s rapid implementation solution spe-
cifically designed to streamline and standardize the im-
plementation process to achieve mission critical business
functionality as soon as possible. ASAP optimizes time,
quality and efficient use of resources of systems imple-
mentation. One of the most important features about
ASAP is that it supports the entire team, both internal
team members from the customer and external consul-
tants: project management, business process consultants
and the technical areas.
In this research, we compare above three methodol-
ogies for characterization of desired one. Major com-
parison issues are shown in Table 3, which are based on
Leem (1999).
With the issues listed in Table 3, the three popular
methodologies are analyzed and compared. Tables 4–6
summarize the results.
2.3. Limits of previous methodology
Architectural and functional limits that previous
methodology possesses are characterized below.
First, they do not provide effective methods linking
business strategy with information strategy planning.
Previous methodologies focus on business strategy is-
sues and their relations while they are leaving it in user’s
discretion how to reflect factors of a business strategy to
information strategy issues planning concretely.

Table 3
Comparison issues for the methodology
Comparison issues Item Description
Functionality Enterprise model Enterprise model elements and their adaptability
Economic valuation Availability on economic valuation
Evaluation on level of information systems Availability on evaluation of IS level
Flexibility Enterprise environment Flexibility on a type of an industry and a scale of an enterprise
Project domain Flexibility on a scale of an IS project
Implementation strategy Flexibility on implementation methods regarding a project
time and implementation scheme
RAD Availability and logical applicability of RAD
Usability Automated tool support Availability and usability of automated tools
Technique support Applicability and volume of techniques
Project management support Availability and applicability of PM tools
252 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261

Table 4
Functionality comparison
Enterprise model
Economic valuation
Evaluation on IS level
METHOD/1 IE-Expert ASAP
Focus on function, data and Focus on data, technology and Focus on data and process
architecture process based on the IEM
Cost and benefit analysis is Cost analysis is achieved on an Tools like Quicksizer are
achieved on an analysis and each stage and a substitutive adopted to suggest an
design stage method may be proposed optimized method
Not provided Not provided Not provided

Table 5
Flexibility comparison
Enterprise environment
Project domain
Implementation strategy
RAD
METHOD/1
The methodology is composed
separately so an environmental
applicability may be achieved
but does not support automated
customization
An applicability may be
achieved but does not support
automated customization
A development and package
introduction are supported
Not provided
IE-Expert
The methodology is composed
separately so an environmental
applicability may be achieved
but does not support automated
customization
An applicability may be
achieved but does not support
automated customization
Not available
Provided by IE/RAD
ASAP
The methodology supports
implementation method
regarding on a scale and a type
of the enterprise by templates
about standard processes on
each industry
Partially provided
Just focused on ASAP package
introduction
Provided by automated
acceleration tools

Table 6
Usability comparison
Automated tool support
Technique support
Project management support
METHOD/1 IE-Expert ASAP
Partially provided Partially provided Process implementation is
supported on a systematic manner
by implementation guide
Fully provided Partially provided Partially provided as a FAQ
Systematic PM method Systematic PM method Systematic PM method is provided
is provided is provided

Second, it is difficult to analyze and validate the
benefit of investment on information systems in a
quantitative manner. Most enterprises agree that enter-
prise hiring advanced information systems can keep
superior competitive power, but there is no solid method
for quantitative valuation about the investment on in-
formation systems. Some researches even suggest that
information systems do not increase competitive power
of an enterprise considering an investment volume on it
(Loveman, 1994). This phenomenon is caused by a de-
ficiency in a systematic and quantitative measurement
method about investment on information systems
(Leem, 1999).
Thus, a quantitative valuation method that analyzes
individual effects(productivity improvement, cost saving
and so on) of information systems and synthetically
estimate its value on an economic aspect is mostly re-
quired (Morihisa et al., 1999; Torkzadeh and Doll,
1999).
Third, they do not support synthetic analysis meth-
ods for automated generation and maintenance of out-
puts from analysis and design. The success of enterprise
information systems largely depends on discovery of
data items and models, on their proper analysis.. These
are considered as a key factor of business goals (Lov-
eman, 1994). Also, success or failure of information
systems is influenced by effective investigation, analysis
and reflection on user requirements. But previous
methodologies support detached methods from analysis
and modeling because they do not include sufficient
modeling techniques. Fourth, they do not support
functionalities for maturity measurement of information
systems. They focus on system development so that they
do not include estimation methods about information
system performance or just include evaluation methods
for operational functionalities. Therefore, measurement
methods which are diversified, objective, systematic and
supporting comparison methods among same kind or
 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261 253

Table 7
Essential requirements for the desired methodology
Issues Essential requirement
Functionality Economic valuation
Maturity (level of information systems) measurement
Effective method linking a business strategy with an information strategy planning
Enterprise model elements (function, information, technology and organization) and their modeling tools
Automated method supporting package software selection
Flexibility Customizable architecture regarding an enterprise environment
Project domain, implementation strategy and RAD
Patterns on a type of industry and scale of enterprise
Variable strategies on development and package software
Usability Automated tools, techniques
Project management

scale of enterprises are required to present a propriety of
investment.
Finally, an automated and systemized method sup-
porting package software introduction is needed for an
effective decision making of users of information sys-
tems. Nowadays most enterprises introduce formal
packages for information systems rather than develop
them. But, from a specific customer point of view,
packages that solves customer’s specific problems and
supports an efficient customization may be very rare.
Thus, a method which have a functionality like vendor
selection, package evaluation and efficient introduction
process can improve effectiveness of information sys-
tems (Mok, 1993).
Based on the observations listed so far, the essential
requirements of the desired integrated methodology are
extracted, Table 7 summarizes them.
3. Framework of the integrated methodology
The proposed integrated methodology framework is
composed of patterns, scenarios (level 1), road map
(level 2), components (level 3) and repository (level 4).
Fig. 1 illustrates a schematic architecture of the meth-
odology. The integrated framework suggested in this
paper are named VIP-2000 (Vision of Information sys-
tems Package – 2000).
Level 1(patterns and scenarios) defines enterprise
specific characteristics and generates a customized ar-
chitecture of methodology. Level 2 (road map) generates
a customized path from planning to running according
to a customized architecture of methodology in level 1.
Level 3 offers detailed functional components needed in
the implementation path generated in level 2, which
includes: information strategic planning methodology
(ISPM), evaluation indices of industrial information
systems (EIII), economic justification measurement
systems (EJMS), support systems for solution intro-
duction & evaluation (S3IE) and unified modeling tool
(UMT). Level 4 (repository) consists of industrial best
practices, database, knowledge storage and consistency
checker.
The methodology also suggests enterprise model ele-
ments of function, organization, information and tech-
nology on the basis of these four levels.
Major characteristics of the integrated methodology,
VIP-2000 are summarized in Table 8.
3.1. Pattern and scenarios
Scenarios define some special cases associated with
various situations in enterprises. The key milestones are
specified according to the scenario defines along with the
special cases. Patterns define enterprise specific charac-
teristics like type of industry, scale of enterprise and
scope of enterprise information systems. The master
road map is converted to the scenario applied road map
through scenarios database, and the scenario applied
road map is converted into the strategy applied road
map through the strategy specific patterns. Finally, the
strategy applied road map is converted to the character
applied road map through the character specific pat-
terns, and it is the fully optimized road map in consid-
eration of characteristic situations of enterprise
information systems. Fig. 2 illustrates these processes.
Patterns are set of classifications based on important
factors of enterprises which need information systems.
Upper patterns accumulate milestones of the method-
ology. Upper patterns are concerned with an imple-
mentation method and strategy. Implementation
methods consists of a development and package intro-
duction. Implementation strategies consists of tradi-
tional approach and rapid application development
(RAD) approach which needs CASE.
Lower patterns accumulate phases, activities and
tasks. It includes types of industry, scale of enterprise
and domain of information systems. This methodology
sorts types of industry by manufacturing industry and
service industry greatly. Types of industry are very im-
254 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261

Fig. 1. The integrated methodology framework.

Table 8
Major characteristics of the integrated methodology
Issues Characteristics
Functionality Economic valuation (Level 3: EJMS component)
Maturity estimation (Level 3: EIII component)
Effective method linking a business strategy with an information strategy planning (Level 3: ISPM component)
Enterprise model elements and their modeling tools (Level 3: UMT component)
Automated method supporting package software selection (Level 3: S3IE component)
Flexibility Customizable architecture regarding an enterprise environment
Project domain, implementation strategy and RAD (Level 1, Level 2:
dynamic architecture of patterns, scenarios and road map)
Patterns on a type of industry and scale of enterprise (Level 1: patterns and scenarios)
Variable strategies on development and package software (Level 2: road map)
Usability Automated tools, techniques (Level 4: repository)

portant element that decide special quality of informa-
tion systems. Therefore, according to types of industry,
implementation methodology of information systems
may be different. A tributary of types of industry to
decide implementation methodology of information
systems is much differences with standard industry
tributary system published in the government. In the
case of manufacturing industry, a standardization of
applications is very difficult because business processes
of each manufacturing company are differ. But, a stan-
 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261
Fig. 2. Role and architecture of pattern and scenarios.
dardization of applications is possible for a service in-
dustry because most business process is resemble and
formal.
This methodology classify the scale of enterprise as a
large one and a small and medium one. Small and me-
dium enterprise is not a tabloid edition of large enter-
prise.
Because small and medium enterprise’s legal term,
market scale, capacity of management, administration
style, organization and financial affairs resources, etc.
are different to be different from large enterprise, im-
plementation methodology must be different too.
3.2. Road map
3.2.1. Architecture
Road map is a set of classification and relation of
tasks needed in life-cycle of information systems’ im-
plementation which ranges from planning to running.
Also, road map is a set of objective functions which
the enterprise must execute successfully to achieve a
competitiveness through information systems and it is
purely related to business environments and imple-
mentation strategy. Therefore, road map presented in
a methodology may be suggested in several paths.
Road map paths in this methodology are provided in
two ways of a development and package introduction.
The development path means that an enterprise de-
velops information system by itself or out-sourced
company. The package introduction path means that
an enterprise buys, installs and runs a packaged
commercial application program. Both road map paths
255
share planning milestone which plans information
strategy, sets a target goal of information systems and
decides whether to develop or introduce a package. In
the case of development, analysis, design and devel-
opment milestones are provided. In the case of a
package introduction, analysis, design, introduction
and customization milestones are provided. And both
paths share a running milestone after an implementa-
tion work.
3.2.2. Road map of development path
Development path consists of planning, analysis,
design, development and running milestones. Planning
milestone analyzes an enterprise’s business environment,
set up a to-be model and plans strategy for information
systems supporting a to-be model. Analysis milestone
divides a to-be model into four modeling elements and
set up detailed to-be models with these elements. And
design milestone makes a technical blue print via these
models. Development milestone sets up and installs
hardware, networks, basic software and develops new
applications. At last, running milestones estimates a
value of information systems, maintain its performance
and upgrades it regarding newly updated requirements
of system users.
On the other hand, package introduction path con-
sists of planning, analysis, design, introduction, cus-
tomization and running milestones. Planning and
running milestone are same as in development path.
Generally, commercial packages include own business
processes, called reference models. Therefore, a to-be
model may be designed via standard business processes,
256 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261
Fig. 3. Components of the methodology.
so that analysis and adoption on reference models are
possible in analysis milestone.
Design milestone focuses on configuration designs of
application functions and user interfaces, and these are
implemented on introduction and customization mile-
stone. Introduction and customization milestone tests
newly introduced systems and its interoperability with
existing information systems, and supports documenta-
tions customized by standard manuals of application.
4. Components
Components consist of five function specific units:
information strategic planning methodology (ISPM),
evaluation indices of industrial information systems
(EIII), economic justification measurement system
(EJMS), unified modeling tools (UMT), support system
for solution introduction & evaluation (S3IE). Fig. 3
summarizes them. Each component must be used
through road map and any independence usage of
component may be possible via scenarios.
4.1. ISPM
4.1.1. Objectives of ISPM
Success of Information Strategy Planning (ISP) de-
pends on the linkage between business strategy and in-
formation strategy (Henderson and Sinfonis, 1988). ISP
consists of strategic management planning, strategic
information system planning and execution planning of
information systems. Control and management of
changes must be conducted to feedback ISP to business
Fig. 4. Flow of ISPM.
strategy. Fig. 4 shows main flow of ISPM (Shelly et al.,
1995).
4.1.2. Key features of ISPM
First, requirement analysis via reviews on documents
and interviews, and evaluation of existing information
systems are conducted in preparation phase.
Second, environment analysis phase executes analysis
of enterprise status, business goals which decides en-
terprise strategy, technical environment, rival com-
pany’s systems and related information technologies. It
estimates enterprise’s competitiveness and level of in-
formation systems. At last, it sets up key strategic points
of information systems.
Third, as-is modeling phase models enterprise in four
modeling elements of technology, organization, infor-
mation and function with simplified terms and symbols
to grasp full states of an enterprise. It verifies the
models’ integrity, and analyses consistency of the mod-
els with business strategy. Finally, it generates im-
 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261 257

provement processes and examines improvement possi-
bility.
Fourth, to-be modeling phase models will be enter-
prise architecture based on improvement process drawn
from as-is modeling. It models goals in four modeling
elements of technology, organization, information and
function. After modeling of each model, it sets up
strategies for implementation of four models and inte-
grates these strategies.
Fifth, value estimation phase executes estimation on
consistency and robustness to judge how faithfully for-
mer phases follow methodology in terms of outputs and
their formality. It estimates achieved competitiveness,
level of information systems and economic values of
information strategy planning from to-be modeling.
Sixth, wrap-up phase gets a final confirmation of in-
formation strategy planning and endorsement of system
users. It also includes training plans for newly adopted
systems and maintenance plans for information strategy
planning.
4.2. EIII
4.2.1. Objectives of EIII
EIII evaluates maturity of information systems. The
evaluation levels that this component provides are di-
vided into four fields, which are IS vision, IS infra-
structure, IS support, IS application & usage (Kim,
1999; Song, 2000).
4.2.2. Key features of EIII
Evaluation levels can be divided into fields primarily
based on relationships between IT and business envi-
ronment. Furthermore, each field is composed of the

Table 9
Levels of IT environment and their fields
Level Field
Subject Relationship
IS vision ISP Business goal, process Comprehensibility
Project plan Technology Availability
Resource, time Suitability
IS infrastructure H/W, network Application Suitability
System S/W User Usability
Supporting tools Provider Satisfaction
IS organization Application
IS support IS direction & Institution H/W, network Usability
User, Provider Supportability
Supporting method IS facilities
Business process
IS application and usage Application User Comprehensibility
User Application Suitability
H/W, network Satisfaction
Supporting team Usability

Table 10
Cost factors of EJMS
Service Labor Overhead cost H/W S/W Conversion
Construc- Application Employment Communication Server cost O/S cost Loss of work during
tion development cost cost cost information systems
introduction
Consulting cost Training cost Public charge PC cost DBMS cost
Equipment cost N/W cost Application
cost
Space cost Peripheral Inefficient work
equipment during the first state
cost

Mainte- Application Employment Communication Articles of Upgrade cost

nance development cost cost cost consumption
Consulting cost Training cost Public charge Machine parts
Equipment cost Exchange cost
Space cost Upgrade cost
258 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261

factors as characteristics of the field, and the factors 4.3. EJMS

have several measurable items, which are practically
observed by evaluators. By using this mechanism, all
environments have to be characterized as measurable
items. Table 9 shows the levels and their fields regarding
IT environment.
4.3.1. Objectives of EJMS
EJMS is a component to present investment effec-
tiveness of information systems to executives through
economical quantification for correct decision-making.

4.3.2. Key features of EJMS

Table 11 Investment of information system for achieving
Major processes of S3IE
business goals must be an investment that can achieve
Stage Sub-stages
maximum effects from limited resources. Thus, EJMS
Strategy Initiation has the goal to supply quantification methodology and
Diagnose
procedure about investment effect of information sys-
Strategy planning
RFP preparation and software evaluation tems. Usual investment propriety analysis on informa-
tion systems consists of economical propriety analysis,
Design Software installation
technological propriety analysis and operational pro-
Customization requirement analysis
priety analysis (Martin, 1989). But, technological pro-
Construction Customization and unit testing priety analysis and operational propriety analysis is not
Integrated testing
so important because most information strategy plan-
Implementation Training ning are based on existing information technology and
Delivery inner resources. EJMS just focuses on economical pro-
Upgrade
priety analysis.

Fig. 5. Architecture of repository.

 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261
Fig. 6. Integrated evaluation structure for the case.
EJMS contains cost factors, benefit factors and in-
fluence factors. Cost factors are classified into by peri-
ods and items. Periods are subdivided into construction,
and maintenance. Items are subdivided into service, la-
bor, overhead cost, H/W, S/W, etc. Table 10 presents
cost factors in EJMS.
4.4. S3IE
S3IE helps decision making of enterprise executives
to plan package introduction strategy, to evaluate each
package and select one. It is based on input data related
each enterprise specific environment. This component
supports whole processes that choose suitable products
in enterprise environment through introduction prepa-
ration, enterprise environment diagnosis, introduction
strategy planning, RFP, proposal document estimation
and package estimation. Table 11 shows the processes of
S3IE.
4.5. UMT and repository
UMT is a modeling tool supporting integration of
outputs through entire life-cycle of implementation of
information systems. UMT efficiently reflects user
requirements and eases system implementation by re-
flecting modeling outputs to system deign and analy-
sis of linkage among modeling. UMT presents tools
as matrix, graph, diagram, report, algorithm and
figure.
UMT supports a repository storing knowledge dat-
abase. It contains industrial best practices, knowledge
storage, database and consistency checker (King and
Teo, 1994). Fig. 5 illustrates the architecture of reposi-
tory.
Best practices are collection of function, information,
technology, organization models describing to-be en-
terprises. Other enterprises can refer these models to
259
improve competitiveness. Consistency checker can
eliminate redundant works and preserve integrity of
data stored in the repository. Database store not only
tools and techniques that UMT offers but also template
of data. Knowledge storage is a repository that collects
knowledge and data generated in the progress of project.
Participants in the project are able to get important
knowledge through the storage. That is, knowledge
storage enables users to get many tips related to the
problems they would face.
5. A case study
5.1. Evaluation framework
This framework follows a systematic movement
through IT results chain (Kim, 1999). Based on the IT
results chain, this case introduces an integrated evalua-
tion framework which is composed of capacity, capa-
bility, and performance. See Fig. 6. The capacity of IS
means physical performance such as speed of CPU,
expertise of provider in the IT area. Capability means
ability that the capacity acts on the business area, and
relationships between components in the IT environ-
ment. The performance denotes the potential energy
that is derived from the IT environment to improve the
competitiveness of the enterprise.
Table 12
Evaluation structure
Structure Definition
Level Components of IT environment
Field Subjects (who) of component, relation-
ships between components and between
IT environment and business environment
Factor Characters of subject and relationship
Item Involved measurable items in factor
260 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261
Fig. 7. Compound indices from the case study.
5.2. Evaluation structure
The measures are derived from IT performance im-
provement model of the integrated methodology, which
is constructed by top-down approach that apply the
concepts of mental model (Bacon, 1992). The measure-
ments structure originated from EIII have four hierar-
chies which are shown in Table 12.
5.3. Results
The Ministry of Information & Communication of
Korea makes use of this component to perform ’Year
2000 evaluation project of information system levels of
Korean enterprises’. Through this project, the usefulness
of the methodology focused on EIII is verified.
117 enterprises have been evaluated. Participants are
following:
• Ownership – Private enterprise: 96, Public enterprise:
21.
• Scale – Large-sized enterprise: 87, Small and medi-
um-sized enterprise: 30.
• Type – Manufacturing: 50, Construction: 13, Financ-
ing: 19, Service: 35.
Average level of all enterprises is upgraded to 58.62
points from 52.84 points of last year. Average level of
upper 30 enterprises is upgraded to 71.33 points from
71.17 points of last year. Average level of IS vision
jumps to 62.94 point from 49.44 points, while average
level of IS support to 63.26 points from 53.32 points.
The case results including above prove practical val-
ues of the methodology in that enterprises adopting
concept and tools of the methodology achieve higher
points in their performance level of IS. Fig. 7 shows
compound indices of special-interested 17 areas, as a
result.
Acknowledgements
This work is supported by the Ministry of Informa-
tion & Communication, Korea.
References
Bacon, C.J., 1992. The use of decision criteria in selecting information
systems/technology investments. MIS Quarterly, September.
Brannback, M., 1993. Effective Strategic Market Management with
Knowledge-based Support Systems. Institute for Advanced Man-
agement Systems Research (IAMSR).
Choi, J., 1998. A framework of the integrated methodology for
industrial information systems. MS Thesis, Yonsei University,
1998.
Delone, W.H., McLean, E.R., 1992. Information systems success: the
quest for the dependent variable. Information Systems Research,
60–95.
Earl, M.J., 1993. Experience in strategic information systems planning.
MIS Quarterly, 1–21.
Grover, V., Jeong, S.R., Segars, A.H., 1996. Information systems
effectiveness: the construct space and patterns of application.
Information and Management 31, 177–191.
Henderson, J.C., Sinfonis, J.G., 1988. The value of strategic IS
planning: understanding consistency, validity, and IS market. MIS
Quarterly 12 (2), 187–200.
Jeong, L., 1997. A study on factors weakens an effectiveness of
information strategy planning. KAIS’97, pp. 217–233.
Mok, K.T., 1993. How engineering consultancies can become more
market oriented. Engineering Management Journal.
Kim, D., Moon, Y., Leem, J., 1998. A study on a evaluation
framework of information strategy planning. Management Science
15 (1), 149–178.
 C.S. Leem, S. Kim / The Journal of Systems and Software 60 (2002) 249–261
Kim, I., 1999. Development of an integrated evaluation system based
on the continuous improvement model of information system
performance. Ph.D Thesis Yonsei University.
King, W.R., Teo, S.H., 1994. Facilitators and inhibitors for the
strategic use of information technology. Information and Man-
agement 27, 71–87.
King, W.R., 1988. How effective is your information systems planning?
Long Range Planning 21 (5), 103–112.
Leem, C.S., 1999. A research on a consulting methodology of
enterprise information systems. ITR Report.
Loveman, G.W., 1994. An assessment of the productivity impact on
information technologies. In Information Technology and the
Corporation of the 1990s: Research Studies. MIT Press, Cam-
bridge, MA.
Lyytinen, K., 1990. Information Systems and Critical Theory – A
Critical Assessment. Department of Computer Science, University
of Jyvaskyla.
Martin, D. et al., 1997. A formal specifications maturity model.
Communications of the ACM 40 (12).
Martin, J., 1989. Information Engineering. Prentice-Hall, Englewood
Cliffs, NJ.
Mahmood, M.A., 1991. A comprehensive model for measuring the
potential impact of information technology on organizational
strategic variables. Decision Sciences 22.
Monheit, M.A., 1990. Information systems architecture: a consulting
methodology. CompuEuro’90. In: Proceeding of the 1990 IEEE
International Conference on Computer Systems and Software
Engineering.
261
Morihisa, H. et al., 1999. A system planning method based on
templates for large-scale manufacturing information systems.
Information and Management 36.
Palvia, P.C., 1997. Developing a model of the global and strategic
impact of information technology. Information and Management
32, 229–244.
Premkumar, G., King, W.R., 1994. The evaluation of strategic infor-
mation system planning. Information & Management 26, 327–340.
Reich, B.H., 1996. Measuring the linkage between business and
information technology objectives. MIS Quarterly 20 (1), 55–81.
Saarinen, T., Scheer, A.-W., 1994. Business Process Engineering.
Springer, Berlin.
Sergas, A.H., Grover, V., 1998. Strategic information systems planning
success: an investigation of the construct and its measurements.
MIS Quarterly 22 (2), 139–163.
Shelly, G.B., et al., 1995. Systems Analysis and Design, second ed.
Boyd & Fraser Publishing.
Song, M., 2000. A study on an evaluation framework of e-business
level. Yonsei University.
Teo, S.H., King, W.R., 1996. Assessing the impact of integrating
business planning and IS planning. Information and Management
30, 309–321.
Theo, J.W., Berghout, E.W., 1997. Methodologies for information
systems investment evaluation at the proposal stage: a comparative
review. Information and Software Technology 39.
Torkzadeh, G., Doll, W.J., 1999. The development of a tool for
measuring the perceived impact of information technology on
work. Omega 27, 327–339.