Ways to Measure Your
Craft Skills Program
Maturity Matrix
Elements LEVEL
LEVEL 11
NOT ENGAGED
NOT ENGAGED
S&K assessment has not been
performed. There is little or
Skill and Knowledge no formal understanding of the
Assessments specific requirements for each
position beyond a rudimentary job
description
The organization has placed
no emphasis on the concept
Task Procedures and
of standard work and has not
"Standard Work"
ANALYSIS
documented any of their critical work
procedures.
Basic craft skill and knowledge gaps
(skill-specific, cross-craft, multi-craft,
Gap Awareness etc.) are not routinely captured and
analyzed for impact with respect to
workforce development training.
The training program developed is
informal, relying largely on publicly
available materials, vendor manuals,
Training Objectives,
and rudimentary photocopies of
Course Outlines and technical information. Applies
Curriculum Maps to new hires, job progression,
apprentice-needs where appropriate,
incumbent craftsmen, refresher
strategies, etc.
In this category, training offerings
may tend to be either heavily on
computer-based learning without
enough hands-on, or self-study, or
Media/Mode lecture, or too generic, or too reliant
on unstructured OJT, etc. Course
Development Plans are not used or
available.
The curriculum consists exclusively
of participant materials and
Instructor Guides & handouts. The materials are
Participant Guides informal and few or no instructor
guides or notes are provided to
ensure consistency of delivery.
The on-the-job training program
is not formally documented
DEVELOPMENT
On-the-Job Training and consists primarily of an
assignment to accompany another
and Job Performance job incumbent for some period of
Measures
time. The expectations from the
training not clear or defined. No
documentation of the process exists.
No job aids are delivered. All of the
learning and reference material is
Job Aids provided in the classroom with little
effect visible on the shop floor.
North America • Latin • America • Middle East • Asia-Pacific
GPAllied Asia-Pac World Headquarters
Level 18, 333 Ann Street 4200 Faber Place Drive
Brisbane QLD 4000 Charleston, SC 29405
o. +61(0)7.3232.1265 o. 888.335.8276
f. +61(0)7.3232.1200 f. 843.414.5779
www.gpallied.com
LEVEL
LEVEL 22
EXPERIMENTING
EXPERIMENTING
Some S&K assessments have been
performed, but the results do not
seem inflected in the design of the
training programs for craftsmen.
Task procedures exist, but they are
informal in nature and consist largely
of copies from owners manuals and
vendor data. It is clear that these
documents do not affect the quality
of work being performed.
Basic craft skill and knowledge gaps
are the focus of the training program
(technical maintenance skills).
The training program has been
largely developed informally, many
without learning objectives defined
for the training interventions.
Although they many not based on
documented learning objectives,
Various training offers are provided
and there is basic management
documentation for craftsmen training
(new hires, advancement, advanced,
predictive training, RCFA, etc.)
There are places where some
training offerings have included a
blended approach of media / modes;
however it seems largely attributable
to individual training providers or
offerings.
When courseware is developed
internally instructor guides are
generally developed, but not
always. Many courses that are
part of the standard catalog do not
have instructor guides - especially
externally developed training.
On-the-job training occurs with
multiple incumbents such that
a variety of view points and
experience levels are leveraged.
No formal structure for the process
exists and the measures of
success are subjective at best. No
documentation of the process exists.
Job aids are incorporated into the
student guides as a reference, but
never appear on the shop floor. It is
clear that these job aids do not effect
the quality of work performed on the
shop floor.
GPAllied EMEA
Guldensporenpark 21-Blok C
B-9820 Merelbeke, Belgium
o. +32(0)9.210.17.20
f. +32(0)9.210.17.28
LEVEL 3
ENLIGHTENED
Formal S&K assessments have
been performed for all craft positions
with the results of this assessment
being fully integrated into the design
of the training program. This was a
one-time significant effort.
The organization has started placing
an emphasis on the development of
standard work and has documented
an extensive library of standard task
procedures.
The organization informally solicits
training needs (beyond basic
craft skills) from employees and
supervisors.
Curriculum (S&K) maps for new
hires and for any craftsmen
progression program are published
and form the basis for the training
programs, designed appropriately
with linkages to the S&K
requirements. Properly developed
learning objectives and job
performance expectations outlined.
Job requirements are tied to
training strategies and performance
objectives, all linked.
Course Development Plans are used
prior to course development These
list the media / mode strategies for
each significant craftsman training
offering. Vendors and in-house
training staff consciously decide
upon the appropriate blend of media/
modes prior to course development.
Instructor guides are developed as
part of the standard training delivery
package - but are seldom used by
the instructors - inconsistencies exist
from instructor to instructor.
The on-the-job training process
is structured and formal with a
task level listing and performance
expectations are defined.
Expectations are a pass/fail with
subjective feedback given informally.
The process is documented with no
follow-on actions - everyone passes.
Job aids are used extensively and
posted on-the-job floor to excess.
It is difficult to determine which job
aids are important. The job aids
are delivered passively on the shop
floor with no active communication
occurring.
LEVEL
LEVEL 44
GOOD PRACTICE
GOOD PRACTICE
Job Task Analysis is the method
of performing skill assessments,
with resultant skill and knowledge
banks used to design the training
programs.
The standard work process is well
developed with a criticality analysis
being applied to identify those
procedures deemed most important.
Formal active training delivered.
Program regularly surveys and
reports gaps and provides training
and intervention to resolve craft
S&K gaps, especially with respect
to future workforce needs, ROI
from cross-crafting, knowledge
capture, while also addresses other
gaps such as RCFA, predictive
maintenance, etc.
Strategies and curriculum maps
are used to guide and manage
the training needs of craftsmen
whether new hire, a craftsman with
a basic level of knowledge wanting
to take the next step, predictive
technologies training, etc. In all
cases, the training interventions
are robustly developed with clear
linkages to the S&K evident in its
development. Properly developed
learning objectives and job
performance expectations have
been defined and are stressed
during delivery.
Course Development Plans are used
both prior to course development
and as part of Continuous
Improvement. These list the media /
mode strategies for each significant
craftsman training offering. The
program utilizes many mixed media
such as video, classroom, CBT, and
printed material.
Instructors are required to own their
Instructor Guides and to ensure
they are up-to-date so that other
instructors can teach the same
course the same way.
OJT formally defined, but peer
feedback process is poor or non-
existent. Performance expectations
defined, but are subjective in nature.
There is a formal documentation
process with candidates required to
improve there skills in certain areas
- not everyone passes the first time
through.
Evidence exists that a robust
process of shop floor reinforcement
of concepts learned in the classroom
being applied via job aids. Active
communication exists regarding
these documents.
Continued on back...
LEVEL
LEVEL 55
BEST PRACTICE
BEST PRACTICE
Job and Task Analyses and S&K gap
assessments are routinely analyzed
to determine training needs and to
anticipate future workforce needs.
The standard work process is well
developed with an appropriate
library of task procedures being
developed and formal active training
being delivered. These standard
procedures are updated and
audited.
A standard process (SDCA) is used
to routinely assess craft knowledge
gaps most critical to business
needs --- includes crafts skills and
knowledge / cross-craft needs,
future workforce needs, and other
domains such as RCFA, TPM,
Predictive Maintenance, Problem
Solving, Planning and Scheduling,
etc. Addresses work environment
factors affecting trainees.
Level 4 plus a complete career
development path defined and
documented for each position to
include not only technical skills,
but interpersonal, leadership, and
problem solving, and analytical
skills where appropriate. Applies
to new hires, job progression,
apprentice-needs where appropriate,
incumbent craftsmen, refresher
strategies, RCFA, multi-crafting, etc.
Blended: The curriculum has been
developed using an appropriate mix
of media to include printed word,
video, instructor-led, self study
modules, hands-on and computer
based training. These media are
available to personnel on demand
with significant guidance and
opportunity for self development
evident.
Instructor guides that drive
consistency in delivery are used to
continuously improve the training
process and to help new instructors
easily learn the required training
standards for craftsmen training
offerings.
The OJT process is well defined
with expectations and peer
feedback occurring based on the
OJT performance. Instructor
expectations for the OJT are
defined. Continual improvement
element to the process clearly
exists. Quantative performance
expectations clearly defined and
implemented.
Job aids are develop and exist on
the shop floor with the proper priority
applied. They are periodically
updated and replaced but they are
actively delivered to the workforce.
It is evident that these job aids
have an effect on the quality of
work performed. Continuous
improvement evident.
 Ways to Measure Your
Craft Skills Program
Maturity Matrix
Elements LEVEL
LEVEL 11
NOT
NOT ENGAGED
ENGAGED
No curriculum based approach is
Individual
applied. Training is delivered in
Development Plans mass to the entire population and
(IDP) & Training is delivered to address the current
IMPLEMENTATION
problem of the day. No strategic
Records
approach.
New courses added to the
Course Validation and curriculum are not reviewed by the
Pilot -Testing job incumbents and no validation or
pilot proof-of-concept process exists.
No dedicated training facility exists
- training is given in any available
Training Environment space - space is largely unsuitable
for training delivery (Noisy,
temperature, interruptions, etc.)
EXECUTION
Instructor qualification is rarely
Instructor
considered. Feedback on
Qualification
instruction is not asked for or given.
Learning Contracts Learning contracts are not used in
any form. The expectations placed
and Performance on those being trained once they
Expectations
return to the workplace are informal.
Examinations or performance
EVALUATION
tests are not given in any form.
The effectiveness of the training
is generally not measured. In
Exam Evaluation this category there may be a
tendency toward written exams
without enough hands-on, practical
examination.
Training Effectiveness Training effectiveness audits not
Audits considered or performed.
North America • Latin • America • Middle East • Asia-Pacific
GPAllied Asia-Pac World Headquarters
Level 18, 333 Ann Street 4200 Faber Place Drive
Brisbane QLD 4000 Charleston, SC 29405
o. +61(0)7.3232.1265 o. 888.335.8276
f. +61(0)7.3232.1200 f. 843.414.5779
www.gpallied.com
LEVEL
LEVEL 22
EXPERIMENTING
EXPERIMENTING
Training maps are developed,
but are not specific to the job
description. The level of support and
coaching provided to the individual
is lacking. Development of skill and
knowledge is discussed and address
infrequently, generally when pressed
by an outside function.
A cursory review of the course
materials is performed by the job
incumbents, but the process is
done informally- likely via email with
limited feedback and no interaction
of the incumbents during the actual
pilot delivery.
Dedicated training facilities exist,
but they are somewhat neglected
and poorly maintained. The training
experience suffers due to the
condition of the training facilities.
Instructor feedback forms are used
for training classes, but they are
informal and consist primarily of
subjective opinions. The results are
not recorded or tracked. No actions
are taken based on the results of
this feedback.
Learning contracts are not used,
but performance expectations of
those attending the training are
verbally delivered at the beginning
of the training session. The direct
supervisors of those being trained
are usually not aware or informed
of specific training expectations or
their role in reinforcing and coaching
learned skills on the job.
Exams are reviewed informally post-
training, but the review is performed
in an informal manner and is not
documented. Corrective actions are
not identified and are rarely followed
up on.
Training effectiveness audits
performed upon return to the
workplace post-training, but they
consist largely of subjective opinions
solely from the participants.
GPAllied EMEA
Guldensporenpark 21-Blok C
B-9820 Merelbeke, Belgium
o. +32(0)9.210.17.20
f. +32(0)9.210.17.28
LEVEL 3
ENLIGHTENED
IDPs are developed to the position
level but are administered poorly.
Some individuals understand and
gain benefit from the IDP, but they
are largely an administrative task
and do not have a significant effect
on the workforce development
efforts.
A formal course review is often
performed for high profile course
deliveries, but is not the standard
operating procedure. Improvement
opportunities for high profile courses
are recorded and followed up on, but
not universally for all courses.
Dedicated training facilities exist,
and periodic upgrades exist over
the years. The facility periodically
becomes outdated and does
not support all of the training
needs beyond simple classroom
instruction. Many interruptions and
derailers to the training experience
exist.
Instructor feedback forms are used
with feedback formally tracked and
corrective actions taken. Forms
consist primarily of subjective
opinions.
The direct supervisors of those
trained attend the opening of
the training session and are part
of the discussion regarding the
performance expectations upon
return to the job.
Examination reviews are performed
and documented, but the follow-up
and corrective action process
is insufficient. Exams remain
largely unchanged through multiple
deliveries of the training sessions.
Training effectiveness audits
performed upon return to the
workplace post-training, but they
consist largely of subjective opinions
provided not only by the participants,
but also from the participant's
managers.
LEVEL
LEVEL 44
GOOD
GOOD PRACTICE
PRACTICE
IDPs are designed to the position
level and are delivered to all
individuals holding that job position.
Some follow-up and coaching is
being done.
A formal review process exists that
is applied to all courses, but the
follow up on opportunities is spotty
and generally a one time effort -
continual improvement not evident.
Dedicated training facilities exist
which are properly designed to
support the optimal training delivery.
Facilities support both classroom,
CBT, and the requisite laboratory
exercises. Minimal interruptions
derailers to the training experience
exist.
The instructor certification
process exists, but is informal
and undocumented and consists
primarily of incumbent or peer
opinions. Feedback forms used,
with corrective actions being taken.
Learning contracts are developed
prior to the execution of the training
and reviewed between the trainee
and their direct supervisor. Clear
and measureable performance
expectations are established for the
trainee upon return to the job.
Formal examinations are provided
with a single version of the exam
existing. Formal exam reviews to
the question level exist. The follow
up process is somewhat informal
and opportunities for improvement
are often missed. No statistical data
on the examination performance is
maintained.
Training effectiveness audits
performed, with a mixture of
subjective and quantitative
performance measures (number
errors, productivity, etc.). The
process to make adjustments to the
training based on the audit results is
spotty and inconsistent.
LEVEL
LEVEL 55
BEST
BEST PRACTICE
PRACTICE
IDPs are designed and delivered to
the individuals within the workforce.
Continual coaching and progress
checks applied. IDPs vary based
on the individual needs and current
skill level.
The course review and approval
process is rigorous and formal. The
pilot session is delivered to the
most qualified job incumbents with
specific and meaningful feedback
provided. This feedback is
incorporated into the course prior to
providing the training to the general
population.
Proper dedicated training facilities
exist with a dedicated training
manager assigned to ensure the
continued upkeep and indicated
improvements to the training facility.
A formal instructor certification
process exists with an individual
task certification process in place
with subject matter experts,
task qualification, and instructor
performance audits being
performed. Corrective actions taken
to address deficiencies.
Learning contracts are used
extensively and are properly
developed (level 4). Follow up
reviews are performed by the
trainers or departmental sponsors
in order to measure the change
in workplace performance, with
a feedback loop existing back to
the development and execution
elements of the model.
Written and hands-on examinations
are provided as part of the training
experience with multiple versions
of the exam existing. Statistical
measures are taken to the question
and specific answer level with
problem areas addressed and
corrected via existing feedback
loops to the exam and training
delivery process.
Training effectiveness audits
performed, with a mixture of
subjective and quantitative
performance measures (number
errors, productivity, etc.). A robust
process to make corrections and
upgrades to the training delivery
based on results exists.