PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 43rd ANNUAL MEETING - 1999 797

MODEL-BASED DEVELOPMENT OF WEB BASED HUMAN PERFORMANCE

MANAGEMENT TOOLS: AN EXAMPLE FROM FINANCIAL SERVICES

Gunilla A. Sundström, Tony Ip, Beth Maddix, Marva Moore,

Prabha Ramakrishnan, Bill Rideout, Yael Yona
GTE Laboratories Incorporated
40 Sylvan Road
Waltham, MA 02454

Abstract: Management of resources in geographically distributed operations

environments such as call centers poses many challenges including: (1) identifying which
measures should be used as key performance indicators, (2) creating a shared view of
performance management across geographical boundaries and enforcing usage of a
common set of performance indicators. The present paper describes results from a multi-
year effort to address these two challenges. A set of models were developed by
leveraging functional modeling concepts proposed in the area of human-machine
systems. These were used to specify the information required and how it must be used in
monitoring and managing call center agents’ performance. These models were the key
building blocks of a user-centered design approach used to develop Web based
performance management tools. A description of modeling process as well as an
overview of the resulting tools and their impact on operational efficiency are provided.

design process. The next section provides a brief description

1. INTRODUCTION
Telephony based call centers are a critical part of business
operations in any services industry, including
telecommunications and financial services. Call centers are
one of the major points of contact for customers and as such
can have a critical impact on customers’ perceptions of a
brand and/or an organization’s ability to provide services. The
primary operational goal of a call center is to provide
customers with what they need, when they need it in a cost-
efficient way. This objective is achieved by using technology
and human resources with about 80% of the overall
operational costs being expended on human resources.
Efficient human performance management is critical not only
to keep customers “happy” but also to ensure operational
efficiency. While excellent discussions about call center
management best practices are available from a practitioner’s
perspective (e.g., Cleveland & Mayben, 1998), few attempts
exist to leverage concepts and lessons learned from the
analysis, design and modeling of other large-scale operations
environments. The primary objective of the present work is to
leverage models and concepts proposed in the context of
analysis and design of decision support for industrial process
control to develop tools to manage outbound call center
activities. Specifically focus is on management of outbound
financial services collections processes. Financial services
collections processes are established to collect money from
customers who fail to pay their bills on time.
In the first section of this paper, the problem domain is
described by highlighting the main characteristics of a
financial services collections process from a systems
engineering perspective. The second section outlines the
approach, namely a model-based user centered analysis and
of the development process and the system architecture. Some
key observations of the impact of using the developed tools in
call centers are highlighted in the last two sections of the
paper. The paper concludes with a brief discussion of the
value of leveraging approaches from other complex system
areas to the development of novel tools to support business
operations.
2. THE FINANCIAL SERVICES COLLECTIONS
PROCESS
A systems engineering view of the collections process is
depicted in Figure 1. The outbound collection process entails
the use of predictive dialer technology (See Figure 1). This
technology is used to automatically call delinquent account
holders. If the call is answered, an agent will try to elicit
immediate payment by credit card or a promise to pay. An
efficient outbound collections process maximizes the number
of promises to pay while minimizing the time call center staff
spend online with customers.
3. THE APPROACH
A model-based user-centered approach was adopted to
identify the performance measures needed and how these
measures should be organized and presented to end-users via
Web based tools. The simple idea being that, as in all
complex systems, a set of models is needed to capture key
characteristics of the system. In particular, focus was on
understanding functional relationships of the system and the
information needed by decision-makers to monitor and
manage behavior of the system.
 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 43rd ANNUAL MEETING - 1999
1 2
Billing Predictive Collections
Dialer System
System
3 4
5
1 Customer payments are posted in the billing system
Delinquent accounts are moved to the predictive dialer
2
Supervisor designs a calling strategy, i.e., identify customers
3 Minimize time
to be called
Predictive dialer calls customer and call is managed by
4 call center staff.
Result is recorded by call center staff in the
5
collections system.
Figure 1. The Financial Services Collections Process
The modeling and design process included the following
key activities:
(1) Extensive on-site visits, including interviews with
management and supervisors, to understand the collections
process. Semi-structured forms were used to organize and
collect the information.
(2) On-site observations and video recording of day-to-day
operations to extract information for the Functional
Information and Knowledge Acquisition (FIKA) model.
FIKA forms were used to structure the on-site observations
(cf. Sundstrom & Salvador, 1995).
(3) Sessions with key staff to develop the model.
(4) In depth analysis of data collected by the systems used in
the collections process including the predicitve dialer and the
collections system.
3.1. The Analysis and Modeling Approach
3.1.1. Leveraging Lind’s multi-level flow modeling
approach. A functional model was developed to model key
operational goals of the basic collections process. This
model was created leveraging Lind’s multi-level flow
798
modeling approach (Lind, 1981). Figure 2 illustrates the
functional model using the concepts of means and goals to
create relationships among operational goals.
Top
Goal: Maximize $’s collected while minimizing costs
Minimize resources
Sub-Goals/ Maximize Maximize used to collect
Means: credit card # of promises
payments to pay
Sub-Goals/ Minimize time spent Minimize time spent to elicit a
Means: to obtain a credit card promise to pay
payment
Sub- Maximize # of times
Goals/Means: accountholders are reached
Sub-
Goals/Means: Maximize time spent calling customers
Sub-
Goals/Means: Minimize time
away from desk
updating accounts
Figure 2: Goal-Means Model Created Using Lind’s Modeling
Concepts
Note that each goal is defined in such a way that it is
associated with the composite behavior of both human and
technology elements within operations. This is the behavior
that needs to be monitored and managed using the Web
application. The present work was focused on the
development of tools to support monitoring and management
of the path highlighted in Figure 2.
3.1.2. Modeling decision making information needs. In
addition to dialing customer’s phone numbers, the predictive
dialer is the primary tool used to track information about the
outbound calling activities. The predictive dialer records
critical information regarding each of the calls placed to
customers. Therefore, an in-depth analysis of the predictive
dialer technology was performed to identify process data. The
identified process data was used to specify a performance
measurement model. This model was developed in
collaboration with collections experts using a combination of
semi-structured interviews and on-site observations. The
measures in the model were classified into three categories: (a)
“Volume” measures, i.e., measures regarding the number of
calls and the amount of time the agents used for various
activities associated with the calls. Example of measures
included in this category are number of connects (i.e., the
number of outbound calls transferred to the agent(s) by the
Mosaix® predictive dialer system during the time period) and
online time (i.e., how long the agent(s) was connected to the
 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 43rd ANNUAL MEETING - 1999
predictive dialer during the time period). (b) “Interaction”
measures, i.e., measures reflecting the results of the customer
interactions. For example., the number of “Promises to Pay”
customers make during the time period is included in this
category. (c) System effectiveness measures, i.e., measures
indicating the behavior of the predictive dialer. While this
performance measurement model identified how data from the
predictive dialer was mapped into performance indicators it
did not specify how the information needs to be used in
operations. As described in Sundström (1997) functional
models can be used to define how information needs to be
used to make various decisions. In this work, the FIKA
modeling approach was used to identify how information
needs to be used to assess whether or not operational goals are
met (Sundström, 1993, 1997). The FIKA model provided the
necessary information to start the design of the web tools.
4. THE WEB TOOLS DEVELOPMENT
4.1. Iterative Prototyping Process
The FIKA model was used to extract initial requirements
and provided the input to a series of prototypes. Each
prototype was reviewed with stakeholders including end-users.
A fieldable prototype was deployed at one call center site to
collect additional requirements using semi-structured
interviews as well as on-site observations. A production
version was developed based on these final requirements.
4.2 The Architecture
An n-tier architecture layering the different types of
enterprise services was used. These services include data,
business, analytical, content and presentation services. Figure
3 provides a functional application overview.
Business
Rules
Services
DB Data Web
Content Server
Services
Services
Analytical
Services Presentation
Services
Figure 3: Web Tools Application Architecture
799
The tools were implemented using primarily Microsoft’s
back-office, COM, scripting (i.e., ASP) and Microsoft’s Web
technology including IIS4.0.
5. FINDINGS
The Web based tools were deployed in a set of
geographically distributed call centers. These Web based
tools were primarily used by the first-line supervisors to assess
performance of outbound collections agents.
After the web-based tools were deployed improvements
in the following were observed in each call center:
(1) Performance measure behavior. Key performance
measures moved in desirable directions. For example, the
number of times an outbound call reached a person
responsible for the delinquent account increased. Thus, the
number of opportunities to elicit a promise to pay increased.
(2) Opportunities to coach staff. One of the major benefits
of the tool is that first line supervisors could spend more time
coaching staff because information required to make decisions
about performance was readily available.
(3) Information Sharing . The use of Web technology
enabled centralized functions to obtain daily insight into
operations at all geographical locations. In addition, the tool
enabled supervisors to share information with their staff and
thus discuss performance issues more objectively.
6. DISCUSSION
Leveraging modeling and analysis techniques developed
in the area of industrial process control proved to be a
critical part of a complex tools development process. A
functional model was developed to model operational
objectives and to achieve agreement among stakeholders
about what the key objectives were. A functional
information and knowledge acquisition (FIKA) model was
developed to specify the information needed to make
assessments related to the various operational objectives. A
model-based user centered iterative design process was
successfully used to develop a complex set of Web based
tools to support human performance management in a
distributed complex operations environment. The deployed
tools demonstrated a very favorable impact on key aspects of
operations in a distributed complex environment.
ACKNOWLEDGMENTS
Gunilla A Sundström is now with Intelligroup Inc., Center of
Excellence for Internet Solutions, 9013 N.24th Avenue, Suite
6, Phoenix, AZ 85021, USA.
 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 43rd ANNUAL MEETING - 1999
REFERENCES
Cleveland, B. & Mayben, J. (1998). Call Center
Management on Fast Forward: Succeeding in Today's Dynamic
Inbound Environment. Annapolis, Maryland: Call Center
Press.
Lind, M. (1981). The use of flow models for automated
plant diagnosis. In J. Rasmussen & W.B. Rouse (Eds.)
Human Detection and Diagnosis of System Failures (pp. 411-
432). London: Plenum Press.
Sundström, G.A. (1993). User modeling for graphical
800
design in complex dynamic environments: Concepts and
prototype implementations. International Journal of Man-
Machine Studies, vol. 38, 567– 586.
Sundström, G.A. & Salvador, A (1995). Integrating field
Work in System Design: A Methodology and Two Case
Studies. IEEE Transactions on Systems, Man and
Cybernetics, vol. 25, no. 3, pp.385-399.
Sundström, G.A. (1997). Designing Support Contexts: Helping
Operators to Generate and Use Knowledge. Control Engineering
Practice, vol. 5 no. 3, pp. 375-381.