|


 

`

 











 






 Best Practice in Terms of ROI

ð Training magazine·s 2003 top 100
companies found that linking training
to specific business metrics requires
evaluation before, during, and after
training initiatives;
ð Many of the Top 100 companies
ranked by Training magazine tie
training initiatives directly to
retention efforts;
ð Also use other metrics ² sales;
production etc;
ð ROI used as basis for continued
improvement of training function;
ð Forecast ROI.
 The Case for ROI ?
Accenture
Allstate Insurance US Department of
Apple Computer Defense
AT&T US Department of Labor
Bristol-Myers Squibb US National Security
British Telecom Agency
Caremark US Department of
Comcast Veteran Affairs
Coors Brewing Co. US General Accountability
Deloitte & Touché Office
Delta Airlines US Office of Personnel
DHL Worldwide Express Management
Federal Express NASA
General Motors
The Value Hierarchy

Paradigm Shift
ROI -Delivery on the
Value Promise
The promise is in the way
we determine
training needs
Moving from input (activity) to
output (results) based training
A Quick Diagnostic
 Ö




 
 

 Absolutely Un True= 1 ; Absolutely True = 5


Link needs to business solutions 1 2 3 4 5

Based on assessment of current vs. desired 1 2 3 4 5

performance

Isolate application and business impact 1 2 3 4 5

objectives
Ensures communication of expectations to 1 2 3 4 5
stakeholders and participants

Ensures the work environment supports 1 2 3 4 5

reinforcement of implementation and
application

Builds partnerships with managers 1 2 3 4 5

Measures at more than reaction and 1 2 3 4 5

learning level

Communicates the results to all stakeholders 1 2 3 4 5

 How Does Your Organization Score

W ic In i i ual reas Do You Nee to ress?

 
8-15
  16-23 2 -31 32-

cti
ity ase o erate
sift Significant rogress ully Integrate esults
 
ittle ocus on esults Or S ift in ertain reas o esults
ase
ase Syste
Activity Results

No established business need Business case

Little relation to performance Focus on improving

performance/change behavior

No/little focus on Demonstrating competence over time

objectives/competencies

No/Little focus on preparing the Prepare environment to ensure

environment transfer and reinforcement

No/Little focus on measuring impact System geared to preparing and

and value proving the case

Mainly input focused Sound balance between input and

output
We need to move towards a results
based approach
But what should we put in place?
What Should Be In Place?

Evaluation approach

Intervention planning
framework

Data gathering approach

Isolation framework

Calculation framework

Lead project
Evaluation Approach
1. Reaction,
Satisfaction
2. Learning
3. Transfer/application
4. Business Impact

5. Return on
Investment

. Intangibles
 Evaluation Approach
Element Description

Reaction Happiness with learning interventions

Learning Changes in knowledge, skills and attitudes

Transfer/application Changes in on the job behaviors over time

(competence)

Business Impact Changes in impact variables

ROI Intervention benefits in relation to cost

Intangibles Intervention benefits in relation to policy

and/or strategic direction and related factors
 Evaluation Approach
Element Value of Case Customer Focus Data Assessment

(relative) Availability Complexity

Administrator
Reaction Limited/Low Consumer Readily Limited/Low

Learning

Application and Business Partner

implementation

Business Impact

ROI
Wide/High Client Research Wide/Complex
Intangibles
Intervention Planning Framework

eneral nee Assess

Learning
Outcomes 
Achieve ? More

Learning
Material

Reaction
Evaluation
Intervention
Planning
Framework

O

 Data Gathering Approach



 
 
  

°
 








 
  



 

  







    

 Isolation Framework

External Factors
Management
Attention

O O
 Incentives
  O
Systems/Procedures


O
Changes
 


Learning
Program 

O


  
  O
 Calculation

Benefit/Cost Program Benefits

Ratio = Program Costs

Benefits Net Program Benefit

X 100
ROI = Program Costs 1