138 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO.
1, FEBRUARY 2012
Assessing Organizational Capabilities: Reviewing
and Guiding the Development of Maturity Grids
Anja M. Maier, James Moultrie, and P. John Clarkson
Abstract—Managing and improving organizational capabilities
is a significant and complex issue for many companies. To support
management and enable improvement, performance assessments
are commonly used. One way of assessing organizational capabili-
ties is by means of maturity grids. While maturity grids may share
a common structure, their content differs and very often they are
developed anew. This paper presents both a reference point and
guidance for developing maturity grids. This is achieved by re-
viewing 24 existing maturity grids and by suggesting a roadmap
for their development. The review places particular emphasis on
embedded assumptions about organizational change in the formu-
lation of the maturity ratings. The suggested roadmap encompasses
four phases: planning, development, evaluation, and maintenance.
Each phase discusses a number of decision points for development,
such as the selection of process areas, maturity levels, and the deliv-
ery mechanism. An example demonstrating the roadmap’s utility
in industrial practice is provided. The roadmap can also be used to
evaluate existing approaches. In concluding the paper, implications
for management practice and research are presented.
Index Terms—Communication, design science, interface man-
agement, maturity grid, maturity matrix, maturity model, new
product development, organizational capabilities, organizational
change, performance assessment, process improvement, project
management, quality management.
I. INTRODUCTION
UALITY management and process improvement initia-
Q tives and their influence on performance excellence have
led to an explosion of standards, regulations, bodies of knowl-
edge, statutes, models, and grids that have been published. These
“improvement frameworks” often seek to enable the assessment
of organizational capabilities. Customers may explicitly require
compliance with some frameworks, and the market may implic-
itly require compliance with others. Some might be imposed by
regulation, others might simply be recognized as being useful in
building or maintaining a competitive advantage [1] or in over-
coming the paradoxical organizational struggle to maintain, yet
replace or renew capabilities [2].
Manuscript received August 19, 2009; revised June 1, 2010 and July 30,
2010; accepted August 25, 2010. Date of publication July 14, 2011; date of
current version January 20, 2011. Review of this manuscript was arranged by
Department Editor J. K. Pinto.
A. M. Maier is with the Department of Management Engineer-
ing, Technical University of Denmark, Lyngby 2800, Denmark (e-mail:
anja.maier@cantab.net).
J. Moultrie is with the Institute of Manufacturing, University of Cambridge,
Cambridge CB3 0FS, U.K. (e-mail: jm329@cam.ac.uk).
P. J. Clarkson is with the Engineering Design Centre, University of Cam-
bridge, Cambridge CB2 1PZ, U.K. (e-mail: pjc10@cam.ac.uk).
Digital Object Identifier 10.1109/TEM.2010.2077289
0018-9391/$26.00 © 2011 IEEE
Irrespective of the driver for adopting an improvement frame-
work, dealing with hundreds of requirements that the diverse
standard-setting bodies impose leaves many companies con-
fused and frustrated. Confusion has triggered calls for an
overview [3] or a taxonomy of improvement frameworks [4], [5].
A taxon suggested by Paulk [5] pertains to management philoso-
phies, such as total quality management (TQM) and associated
with it maturity assessment grids. Maturity grids can be used
both as assessment tools and as improvement tools. Crosby’s
quality management maturity grid (QMMG) features as the pi-
oneering example, advocating a progression through five stages:
uncertainty, awakening, enlightenment, wisdom, and certainty
(see Fig. 1).
In case of a voluntary evaluation of performance levels, com-
panies often look for assessments that do not take too long and
do not cost too much, which makes maturity grid assessments
especially attractive. However, while managing organizational
capabilities is a significant and complex issue for many com-
panies and features prominently in organization literature, we
nevertheless observe that the contribution of assessments using
maturity grids has so far been overlooked in academic litera-
ture. It seems as if maturity grids have been in the shadow of the
more widely known capability maturity model (CMM) [6], [7]
and its derivatives, including the capability maturity model in-
tegration (CMMI) [8] and the people-CMM [9]—all developed
at Carnegie Mellon’s Software Engineering Institute (SEI).
A. Connection Between Maturity Grids and Models
Differentiating between capability maturity grids and CMMs
is difficult. While they are complementary improvement frame-
works with a number of similarities, key distinctions can be
made with respect to the work orientation, the mode of assess-
ment, and the intent.
1) Work Orientation: Maturity grids differ from other pro-
cess maturity frameworks, such as the SEI’s CMMI, which ap-
plies to specific processes like software development and acqui-
sition. The CMMI model identifies the best practices for specific
processes and evaluates the maturity of an organization in terms
of how many of these practices it has implemented. By contrast,
most maturity grids apply to companies in any industry and do
not specify what a particular process should look like. They
identify the characteristics that any process and every enterprise
should have in order to design and deploy high-performance
processes [10].
2) Mode of Assessment: Typically, an assessment using the
CMMs consists, among other instruments, of Likert or binary
yes/no-based questionnaires and checklists to enable assessment
of performance. In contrast, an assessment via a maturity grid is
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
Fig. 1. Excerpt reproduced from the QMMG [88].
typically structured around a matrix or a grid. Levels of maturity
are allocated against key aspects of performance or key activi-
ties, thereby creating a series of cells. An important feature of a
maturity grid approach is that in the cells it provides descriptive
text for the characteristic traits of performance at each level,
also known as a “behaviorally anchored scale” [11].
3) Intent: Many CMMs follow a standard format and are
internationally recognized. As a result, they can be used for
certification of performance. Maturity grids tend to be some-
what less complex as diagnostic and improvement tools without
aspiring to provide certification. Accordingly, the intention of
use by companies differs. Companies often follow a number of
approaches in parallel and a maturity grid assessment may be
used as a stand-alone assessment or as a subset of a broader
improvement initiative.
B. Lack of Cross-Domain Reviews
There is a lack of concerted cross-domain academic effort in
understanding the limitations and benefits of maturity grids. In
specific knowledge domains, there have been some studies that
compare a variety of maturity assessments, mostly focusing on
maturity models: Becker et al. [12] compared six maturity mod-
els for IT management, Rohloff [106] reviews three maturity
models for business process management, Kohlegger et al. [13]
conducted a qualitative content analysis of a set of 16 maturity
models, DeBruin and Rosemann [14] presented a cross-domain
review of two maturity models for knowledge management and
for business process management, respectively, Siponen [15] ex-
plored three evaluations based on maturity principles in the area
of information management, and Pee and Kankanhalli [16] com-
pare maturity assessments for knowledge management. Analy-
ses concentrate mostly on a small sample of maturity models
and describe strengths and/or weaknesses of existing approaches
within the respective domains, i.e., IT management, knowledge
management, business process management, and information
management.
Studies with a larger sample size that use maturity assessment
methods to gauge the current level of a specific organizational
capability in different industry sectors have been conducted, e.g.,
by Ibbs and Kwak [17], Pennypacker and Grant [11], [18], and
Mullaly [19] in the domain of project management. These stud-
ies report on a substantial interest in the development of viable
methods to assess and improve project management maturity
139
and recognize the need for synthesis among maturity assess-
ment methods.
Each of the previous research efforts cited succeeded in com-
paring maturity models in specific knowledge domains or used
specific maturity assessment methods to arrive at an overall
level of, for example, project management within a number of
industry sectors. As such, these efforts represent valuable initial
steps to give researchers and industry the needed overview in
their quest for awareness and perhaps synthesis of assessment
methods.
Given what has been said earlier, there is a need for a cross-
domain review of maturity grids and a gap in current understand-
ing relating to underpinning theoretical concepts of maturity-
grid-based assessments. The consequences of these are that new
methods are developed from unsuitable foundations or that un-
necessary effort is often expended in developing new assessment
tools that duplicate those that already exist. Therefore, this pa-
per aims to review existing maturity grids to provide a common
reference point.
C. Lack of Guidance for Authors of Maturity Grids
Since the early 1990s and in parallel to the absence of an
inter-domain academic debate, we see rapid growth in the de-
velopment and use of maturity grid assessments in management
practice across a number of industry sectors. Such maturity
grids are developed by researchers in academia, practitioners,
and consultants, and as a result, these are often proprietary or
difficult to access. When examples do reach the academic liter-
ature, understandably, they are published in specialized journals
relating to the domain addressed. With few exceptions, work is
presented without reference to that which precedes it and new
language is developed for concepts that have already been de-
scribed elsewhere. Many such efforts have been criticized as ad
hoc in their development [20]. It is perhaps not surprising, as in
the absence of guidance, authors do what they think is best and
that is often not good enough, especially considering the poten-
tial impacts of assessment results on a company’s operations and
employees’ morale. Consequently, this paper aims to suggest a
more rigorous approach to the development of maturity grids.
Recent studies intend to aid the development of maturity
models. Becker et al. [12] compare six maturity models for
information technology management and suggest a procedural
model for their development, Mettler and Rohner [21] suggest
140 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
decision parameters for the development of maturity models in
information systems, Kohlegger et al. [13] compare the content
of 16 maturity models, and Van Steenbergen suggest a set of
questions for subsequent development of maturity models. Van
Steenbergen et al. [105] propose parameters for the develop-
ment of focus area maturity models. This paper complements
these studies by reviewing existing maturity grids, by focusing
on underpinning theoretical concepts of maturity-grid-based as-
sessments and by guiding their development and evaluation.
D. Objectives of the Paper
The objectives of this paper are to review existing matu-
rity grids to provide a common reference point and to suggest
the parameters for a more rigorous approach to the develop-
ment of maturity grids. This paper is intended for practitioners
in industry and academic researchers concerned with process
improvement, intervention, and change management in orga-
nizations. Both readerships might benefit from an overview of
published maturity grids for assessing organizational capabil-
ities, and potentially be introduced to grids that they had not
previously seen. Furthermore, they might benefit from guidance
for the systematic development of their own maturity grid.
E. Structure of the Paper
The remainder of the paper is organized as follows.
Section II describes the methods used to elicit the content of
this paper. Section III introduces the notion of maturity and
traces its history and evolution. In Section IV, existing maturity
grids are reviewed. Section V suggests a roadmap as a process
for creating maturity grids. Section VI presents an illustrative
example of its application. Section VII concludes the paper with
the main implications for management practice and research.
II. METHODS
This section explains the rationale for selecting the review
sample of this paper and furthermore describes how the sug-
gested roadmap for the development of new and the evaluation
of existing maturity grids was built up and evaluated.
A. Selection of the Review Sample
Our sampling strategy for the review consisted of the follow-
ing activities: First, we keyword-searched academic databases
and World Wide Web search engines. Second, we followed-up
references from publications found in the preceding activity. Fi-
nally, to check comprehensiveness and gather suggestions from
experts in the field of organizational studies, we sent the list
of 61 grids and models to five academic scholars active in this
field. Out of the list of potential models, we selected maturity
grids for further analysis that fulfilled the following criteria.
1) Grid-based approach: The grids need to be akin to the
original QMMG by Crosby as opposed to following a
staged CMM-style. The representation in grid or matrix
format using text descriptions as qualifiers is either used
as underlying model and/or assessment instrument. Here,
no differentiation is made between a grid, a matrix, and a
table [22].
2) Self-assessment: A number of assessment methods use
an external facilitator to assign and/or decide on scores
based on participants’ comments and/or require a certified
auditor. These approaches do not meet this criterion as this
paper focuses on approaches aimed at self-assessment.
3) Publicly available: Many maturity grids are proprietary
tools generated by consulting organizations. Approaches
included in this review are all in the public domain and
available without cost.
Table I summarizes the 24 maturity grid assessments ana-
lyzed in this study, presented in chronological order of pub-
lication. Our sample consists of maturity grids developed by
academic researchers, industry, and consulting firms, provided
they meet the aforementioned criteria. As a result, many mod-
els that make significant contributions in their own field were
not included in this review. To mention a few, for example, the
Achieving Excellence Design Evaluation Toolkit [23] uses a
Likert-scale and is, as defined in this paper, not a grid-based
approach. Kochikar’s approach in knowledge management [24]
is particularly influenced by the CMM in that it is a staged
approach, requiring baseline performance in certain key re-
search areas before advancing to the next level of maturity. Fur-
ther, Ehms and Langen’s comprehensive model [25] in knowl-
edge management relies on an auditor to infer the scores from
participants and the learning organization maturity model by
Chinowsky et al. [26], for example, violates the first criterion as
it is not a grid-based tool.
Outside the scope of the paper is also work on related terms.
For example, first, technology readiness [27], [28] and uptake
of its principles, e.g., system readiness [29] or organizational
readiness [107]. Second, life-cycle theories, for example, de-
scribing the adoption of a new product, technology, or service
in the market, often visualized using S-shaped curves [30], [31];
the development of electronic data processing toward maturity
(using six stages of growth e.g., [32]) and the phases of growth
organizations, in general, pass through toward maturity in the
market [33].
B. Elicitation of Guidance
The overall research approach taken for the review and sug-
gestion of subsequent guidance is that of conceptual analy-
sis [15]. Individual maturity grids are compared according to
critical issues: the assessment aim, the selection and rationale of
process areas, the conceptualization of leverage points for matu-
rity, and administration of the assessment. In order to show how
we came to a particular conclusion, relevant citations from the
original material are included wherever possible. The roadmap
takes the form of a description of the sequence of four process
steps [34] with a set of decision points for each phase. Develop-
ment of the roadmap was pursued like a design exercise in that it
aimed to create an innovative artifact—a method—intended to
solve an identified problem. The underlying theoretical perspec-
tive is thus design science [35], [36]. For presentation of this
paper, we have furthermore been inspired by Eisenhardt’s [37]
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS 141

TABLE I
COMPARISON OF EXISTING MATURITY GRIDS
142 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012

TABLE I
(Continued)
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS 143

TABLE I
(Continued)
144 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012

TABLE I
(Continued)
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
roadmap to develop theory from case studies, which alerts the
reader to steps and associated decision points in the development
journey.
Guidance, in the form of a roadmap with four phases and a
number of associated decision points, was developed in three
steps: first, more than 20 extant maturity grids for assessing
organizational capabilities were reviewed. The sample contains
contributions resulting from the fields of management science,
new product development, engineering design, and healthcare.
Second, guidance was elicited on the basis of experience from
the authors of this paper. Third, two experts, who have inde-
pendently developed and applied a maturity grid for assessing
organizational capabilities were interviewed. Both experts have
undertaken consulting work and are now pursuing academic
careers in engineering and construction. Interviews were con-
ducted to obtain further insights and to validate our results from
comparing extant approaches in literature and findings from our
own experience with developing and applying maturity grid as-
sessments in small- and medium-sized companies as well as
large multinational corporations from a number of industry sec-
tors. In summary, insights from reviewed literature, the authors’
own experience and the experts’ feedback were used as the basis
of this guide.
C. Evaluation of Guidance
The roadmap suggested in this paper was evaluated in two
ways. First, application of the roadmap’s use in industry demon-
strates its workings. Outcomes of multiple applications of the
maturity grid to assess communication at departmental inter-
faces given as the case example in this paper is taken as
an indicator for both the roadmap’s and the grid’s reliabil-
ity. In addition, feedback from the participants in the assess-
ments is also taken as direct member validation [38] of the
communication grid method (CGM) [71] and as indirect ev-
idence of the roadmap’s utility. Second, in connecting to the
design science perspective, a further evaluation of the de-
velopment method presented here is provided by applying
the guidelines for devising an artifact formulated by Hevner
et al. [36] and reformulated and used specifically in the context
of requirements for maturity models by Becker et al. [12] (see
Section VI and Table III).
III. MATURITY
When discussing the concept of maturity, it is important to
provide definitions as the language can be inconsistent and con-
fusing. Even within one field of expertise, the concept of ma-
turity espoused might not be one and the same. This section
introduces a dictionary definition of maturity and continues by
specifying what in the literature has come to be termed “pro-
cess maturity,” “organizational maturity,” “process capability,”
“project maturity,” and “maturity of organizational capabilities.”
A. Notion of Maturity: A Dictionary Definition
Broadly speaking, there is reference to two different aspects
of the concept of maturity. Firstly, there is reference to some-
thing or someone as having reached the state of completeness
145
in natural development or growth, in other words, the “state of
being complete, perfect, or ready”. Secondly, there is reference
to the process of bringing something to maturity, “to bring to
maturity or full growth; to ripen” [39].
It is the latter definition, which stresses the process toward
maturity that interests us in the paper. How do authors of indi-
vidual maturity grid assessments conceptualize a progression to
maturity? Searching for answers to this question, one realizes
that the concept of maturity is best discussed in connection with
the context within which it has been applied.
B. Evolution of the Notion of Maturity in Literature(s)
The concept of maturity has seen widespread attention in
a number of academic fields. While maturity grids have been
known for some time, their popularization as a means of assess-
ment has been more recent [19].
1) Process Maturity: The concept of process maturity stems
from TQM, where the application of statistical process control
techniques showed that improving maturity of any technical and
business process ideally leads to a reduction of variability in-
herent in the process, and thus, an improvement in the mean
performance of the process. While Crosby has inspired the no-
tion of progression through stages towards maturity, his maturity
concept as a way of measuring organizational capabilities was
not formalized [5].
2) Organizational Maturity: Through the widely adopted
CMM for improvement of the software development process
(CMM-SW), this concept of process maturity migrated to a
measure of organizational maturity. Integral to the CMM-SW
is the concept that organizations advance through a series of
five stages or levels of maturity: from an initial level to a
repeatable, defined, managed, and an optimizing level. These
levels describe an evolutionary path from ad hoc, chaotic pro-
cesses to mature, disciplined software processes, and define
the degree to which a process is institutionalized and effective
[6], [40].
3) Process Capability: Rather than measuring organiza-
tional capability with a single value, ISO/IEC 15504 measures
process capability directly and organizational capability with a
process capability profile. CMMI integrated both organizational
maturity and process capability. Although ISO/IEC 15504 and
CMMI both use capability levels to characterize process capa-
bility, their operational definitions are somewhat different. The
key taxonomic distinction is between a multilevel organizational
versus process measure.
Paulk [5] suggests the term organizational capability to char-
acterize a hybrid between organizational maturity and pro-
cess capability. This is different from the notion of organi-
zational capabilities applied in this paper. In addition, irre-
spective of the way of arriving at an overall score either on
the level of processes (process capability) or aggregate level
for an organization (organizational maturity), the notion of
higher levels of maturity representing increased transparency is
retained [5].
4) Project Maturity: Perhaps since software is developed
through projects, it is natural that the concept of organizational
maturity would migrate from software development processes
146 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
to project management, and this has been reflected in an inter-
est in applying the concept of “maturity” to (software) project
management [17], [41]–[46]. Although the focus on assessing
project management using the notion of maturity has started
comparatively recent, a number of alternative models have been
released [18], [19], [44], [47].
While inspired by CMM- or ISO/IEC 15504-like notions of
maturity, which focus on process transparency and increased
(statistical) control, research into project management maturity
shows variations in how maturity is conceptualized. One way to
determine a mature project is by looking at what organizations
and people are doing operationally [17], [48]. Skulmoski [49]
goes further to include an organization’s receptivity to project
management. Andersen and Jessen [47] continue in this vein
and determine maturity as a combination of behavior, attitude,
and competences. Maturity is described as a composite term,
and characteristics and indicators are used to denote or measure
maturity. In response to the many competing models, the Project
Management Institute (PMI) launched the organizational project
management maturity model (OPM3) program [18] as a best
practice standard for assessing and developing capabilities in
portfolio management, program management, and project man-
agement [50].
5) Maturity of Organizational Capabilities: The concept of
capability has been used in strategic literature, specifically in
the resource-based view to explain differential firm perfor-
mance [51]–[54]. The capability perspective hinges on the in-
ductively reasoned relationship between process capability and
business performance. This paper uses the terms organizational
capabilities as the collective skills, abilities, and expertise of
an organization [55], [56]. In this vein, organizational capa-
bilities refer to, for example, design, innovation, project man-
agement, knowledge management, collaboration, and leader-
ship [55]. Thus, organizations can be viewed as comprised of a
set of capabilities [57], which are the skill sets that provide an
organization with its competitive advantage.
While it seems potentially misaligned to have first described
process maturity, followed by organizational maturity, and fi-
nally, by one example of an organizational capability, project
management, before finally moving to the focus of this paper,
maturity of organizational capabilities, in general, two reasons
justify this sequence. First, the historic timeline of influence is
maintained. Second, this body of literature engages with and
shows variations in conceptualizations of maturity that would
be fruitful across disciplines. Variations show that there is more
than one leverage point to achieve maturity—the subject of the
cross-domain review of existing maturity grids in the ensuing
section.
IV. REVIEW OF UNDERLYING NOTIONS OF MATURITY IN
EXISTING MATURITY GRIDS
This section focuses on the notion of maturity across the
selected sample of maturity grids. Furthermore, Table I gives an
overview of extant maturity grids and allows comparison with
respect to the aim, the scope, the administration mechanism,
the selected process areas, and most importantly, the selected
maturity levels.
A. Visualization of Maturity Levels
A common principle is to represent maturity as a number
of cumulative stages, where higher stages build on the require-
ments of lower stages. The practice, with the highest number
representing high maturity and the lowest number representing
low maturity, appears to have wide practical acceptance.
This evolution toward maturity has been visualized in a num-
ber of ways, e.g., using a ladder representation [47] or using a
spider web representation [46]. Either way, the different steps
on the ladder or the different intersections on the spider web
have to be characterized and more often than not opinions about
step differences diverge. Even when focusing on one application
area without close examination, the assumption cannot be made
that maturity levels in different maturity grids describe exactly
the same concepts.
B. Review of Underlying Notions of Maturity
What makes organizational capabilities mature? There are
many possibilities to answer this question, where each answer
is based on a certain rationale. Such rationale is usually, whether
explicitly stated or implicitly embraced, a statement about lever-
age points envisaged to be used in organizational change initia-
tives.
Directly to compare structural differences and theoretical as-
sumptions about (organizational) maturity and conceptualiza-
tions of organizational change, excerpts of five grids concern-
ing “coordination” and “collaboration” are selected. We discern
four leverage points that have been used: 1) existence and ad-
herence to a structured process; 2) alteration of organizational
structure; 3) emphasis on people; and 4) emphasis on learning.
1) Existence and Adherence to a Structured Process (Infras-
tructure, Transparency, and Formality): A number of extant
grids base the selection of maturity levels on the systematic
process improvement approach, underlying the five-level rank-
ing system introduced by the SEI, e.g., [11], [17], [20], [48].
Maturity is defined as “[. . .] the extent to which a specific pro-
cess is explicitly defined, managed, measured, controlled, and
effective” [6]. Thus, maturity is defined as the degree to which
a process is institutionalized and effective [40].
Organizations are encouraged to use existing and well-known
methods and practices to progress along the maturity scale. The
assumption is that the more structured a process and the more
transparent in terms of measurability of performance the better.
The levels range from level 1 “initial,” to level 2 “repeatable,”
to level 3 “defined,” to level 4 “managed,” and level 5 “opti-
mized,” where the lowest maturity level 1 corresponds to initial
or learner, and the highest maturity level 5 corresponds to a de-
sired performance of a process of best practice. In software, for
example, this translates into continuous improvement through
focused and sustained effort toward building a process infras-
tructure. It is often assumed that at the higher levels of maturity,
the process is no longer needed, as the “right” performance is
embedded (see also [43] and [58]).
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
Fig. 2. Redrawn “teamwork” by Hammer [10].
Fig. 3. Redrawn activity 7—Coordinating R&D and marketing from Szakonyi [59].
2) Alteration of Organizational Structure (e.g., Job Roles
and Policy): One structural element of how people are envis-
aged to work together in an organization is through the use
of teams. Depending on the culture of the organization, team-
work might involve people from one discipline, it might be
cross-disciplinary, it might involve customers and/or suppliers,
it might be different from project to project or it might happen in
a standardized way. Hammer’s process audit, for example, [10]
sees maturity of teamwork to evolve from “project focused” to
“cross-functional,” to “teamwork as norm” through to “involv-
ing customers and suppliers” (see Fig. 2).
For Szakonyi [59] (see Fig. 3), maturity seems to be an in-
crease in knowledge about skills, methods, and responsibilities.
The most mature form of collaboration between, in this case,
R&D and marketing, seems to be a technical person in-charge
of marketing. This imposes a specific form of organizational
model as an instantiation of best practice. The description is
“static” and does not indicate what aspects lead to improve-
ment. In fact, the job role is more related to responsibilities.
It could be inferred that organizational change with regard to
coordination of teams is best initiated via structural changes in
job roles and training in skills, and methods—which makes it
also a candidate for a focus on people (see Section IV-B3).
3) Emphasis on People (Skills, Training, and Building Re-
lationships): A number of maturity grids conceive of people
as the best leverage points for an evolution toward maturity of
collaboration and cooperation (e.g., Fraser et al. [73] and Con-
structing Excellence [60]). Fraser et al. use a number of bullet
points for each text description addressing different aspects (see
147
Fig. 4). For collaboration strategy, within level 2 (occasional ad
hoc partnering), one bullet point reads “no agreed policy.” Under
level 3 (established partners), we find a bullet point that reads
“internal skill shortages clearly recognized.” Reading through
the cell descriptions from the lowest to the highest level of
maturity, the underlying assumption is that successful collabo-
ration is fostered through long-term relationships, and structured
and strategic relationship planning, most importantly incorpo-
rating external partners into the core team (see Fig. 4).
Text descriptions in Constructing Excellence [60] (see Fig. 5)
embrace the underlying notion that “the more interchange and
participation is practiced among teams the better,” regardless
of the specific task. Further, “lack of trust” and “power strug-
gles” are placed at the lowest level of maturity. The underlying
assumption seems to be that organizational change could be
successful by focusing on interventions in the social relations
among employees, in contrast to structural changes, as we have
seen earlier.
4) Emphasis on Learning (Awareness, Mindset, and Atti-
tude): Strutt et al. [61] and Maier et al. [62] operating in very
different application areas have chosen to adapt ideas from
the concept of single- and double-loop learning [63], [64] in
order to discriminate between the maturity levels. These au-
thors have a shared rationale, but operationalization is differ-
ent. Strutt et al. [61] chose to define the text descriptions in a
prescriptive way, Maier et al. [eg., 62] in a descriptive manner.
Maier et al. (see Fig. 6) chose an underlying maturity con-
cept progressing toward raising awareness for adequate actions
and attitudes. The underlying notion of change seems to be that
148
Fig. 4. Redrawn “collaboration strategy” by Fraser et al. [73] (General discussion questions omitted).
Fig. 5. Redrawn “collaboration” by Constructing Excellence [60] (direction of rows changed).
Fig. 6. Redrawn “collaboration” by Maier et al. [89].
proactive collaboration (Level C) is favored over reactive (Level
B). Ultimately, mapping of the current and desired situations is
aimed for irrespective of the specific levels.
In summary, the aforementioned analysis shows how the same
subject is conceptualized in different ways. Cell descriptions
“reveal” the researchers’ views of an organization, its processes,
people, and products. Conceptualizations impact organizational
change initiatives as they specify leverage points. While some
maturity grids can be clearly placed, others, e.g., Szakonyi [59]
use a mixture of organizational structure and emphasis on people
as leverage points for capability improvement.
V. ROADMAP: PHASES AND DECISION POINTS
In order to develop sound maturity grids, we suggest a
roadmap consisting of four phases: planning, development, eval-
IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
uation, and maintenance, as depicted in Fig. 7. For each phase,
a number of decision points and decision options are discussed.
While these phases are generic, their order is important. Pro-
gression through some phases may be iterative and earlier work
might have to be re-visited. Although courses of action with re-
spect to decision points within the phases of this roadmap may
vary, the phases are consistent. Consequently, they lend them-
selves to being applied across multiple domains. The follow-
ing sections describe each of these phases, decision points and
options in more detail. Although predominantly aimed as guid-
ance for future developments of maturity grids, the roadmap
may be used for evaluative purposes of existing assessments.
Given differing contexts of development and application, dif-
fering performance goals, and perspectives on organizational
development and change, the subject matter does not lend it-
self to all-encompassing prescriptions and specific instructions
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
Fig. 7. Phases and decision points of roadmap to develop new and evaluate existing maturity grids.
for navigation among different options. Instead, the roadmap
alerts researchers and practitioners to a set of decisions, which
need to be taken and made explicit when developing a maturity
grid.
A. Phase I—Planning
Phase I, the planning phase, sees the author of a maturity
grid decide on the intended audience (user community and im-
provement entity), the purpose of the assessment, the scope, and
success criteria. Each decision point will be described in turn.
1) Specify Audience: The first important decision when de-
veloping maturity grids is to specify the work orientation, i.e.,
who the expected users are. The term audience refers to all
stakeholders, who will participate in various aspects of the as-
sessment, be it for the data acquisition process, as implementer
of results, or as the subject of the assessment, also known as the
improvement entity. Multiple audiences may be addressed for
varying reasons. A quality manager or product development en-
gineer, for example, might be the target audience for providing
information on the capability to be assessed. The improvement
entity, however, could be the whole research and development
department. Further, the whole assessment exercise might be
aimed to provide recommendations for the Chief Executive Of-
ficer’s corporate planning.
For reasons of clarity and accuracy of interpretation, it is
necessary to differentiate between audiences, e.g., the users
and the improvement entity. Decisions will have both logisti-
cal and conceptual implications. Logistical implications con-
cern predominantly time and resource constraints relating to
the participants and facilitator of the assessment. Conceptual
implications relate mainly to validity, reliability, and generaliz-
ability of the assessment and address questions, such as: can one
person in the company judge or decide alone for the company
in question? Can results acquired from one group of employees
be transferred to hold true for a different group?
2) Define Aim: Categorizations of software improvement
initiatives seem to distinguish between two “improvement
paradigms”: an analytic and a benchmarking one [65], [66].
149
The analytic one aims for evidence to determine what improve-
ments are needed and whether an improvement initiative has
been successful. It includes general management philosophies,
such as Crosby’s TQM, which are general principles applicable
in almost any context. The benchmarking one depends on identi-
fying best practices and/or an “excellent” organization in a field
to which practices are compared. It specifies best practices that
have been demonstrated to add value in a particular context and
have been explicitly stated in models and standards, of which
improvement based on ISO 9001 or CMMI are examples. An-
alytic and benchmarking strategies can be complementary [5].
Both may lead to a measurement-based strategy, which means
that processes (and systems) are measured and compared to ob-
jectives set by management or industry standards in order to
identify which processes need to be improved.
Even though, accordingly, maturity grids are analytic strate-
gies, it is evident that in existing examples two overarching
aims can be identified; first, improvement through “raising
awareness,” and second, improvement through “benchmarking”
across companies or industry sectors. Benchmarking includes
comparison against an identified best practice example and mak-
ing statements about performance of a whole industry sector in
terms of a certain process or capability. Benchmarking seems
to incorporate “raising awareness,” but not vice versa. In order
for a grid to be used to benchmark processes and capabilities
across an industry sector, it must be applied to a high number of
companies with similar parameters to attain sufficient data for
valid comparison.
3) Clarify Scope: An author needs to determine the scope
of the grid. Is it designed to be generic or domain-specific?
For example, is a grid developed to assess and improve energy
management in general or in a particular discipline, e.g., con-
struction? If it is supposed to be domain-specific, it is especially
important to gather information about the context, the idiosyn-
crasies and terminology of the specific domain in order for it to
be understood by and of relevance to the audience.
4) Define Success Criteria: How do authors of maturity
assessments know whether development and application was
successful? One way would be to check whether success
150 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
criteria have been fulfilled. Success criteria need to be deter-
mined at the outset and manifest in the form of high-level
and specific requirements. Assessment methods are intervention
methods. High-level requirements for an intervention method
developed in managerially focused action research [67] are, for
example, usability and usefulness. Usability mainly addresses
the degree to which users understand the language and concepts
used [68]. Usefulness could be seen in terms of companies’
perceptions of whether they found the assessment helpful in
stimulating learn effects or in leading to effective plans for im-
proving a certain situation. Specific requirements pertain to the
individual context and are also influenced by the underlying the-
oretical stance used by the author of a maturity grid (see Phase
II). The requirement list needs to incorporate the developer’s
and the user’s objectives.
B. Phase II—Development
Phase II, the development phase, defines the architecture of
the maturity grid. The architecture has a significant impact on
its use. An author makes decisions about the process areas to
be assessed, the maturity levels (rating scale) to be assigned,
the cell descriptions to be formulated, and the administration
mechanism to be used.
1) Select Process Areas (Content): Selecting process areas is
arguably one of the most difficult aspects of devising a maturity
grid. The goal is to establish key process areas that are mutually
exclusive and collectively exhaustive. An effective assessment
should be based on an underpinning conceptual framework,
generated from (traceable) principles of good practice [69]. In-
evitably, the selection of process areas yields insights into the
authors’ conceptualizations of the field. The conceptual frame-
work underlying the assessment method determines the scope
of the assessment.
In a number of existing grids, justification of process ar-
eas is based on the experience in the field of the origina-
tor [10], [59], [70] and by reference to established knowledge in
the field, e.g., in the domain of project management, the PMI’s
knowledge areas are referred to (for example, as in Grant and
Pennypacker [11]). In the absence of significant prior experi-
ence, and in a relatively new field, it may not be possible to
gather sufficient evidence through existing literature in order to
derive a comprehensive list of process areas. In this instance, a
literature review is considered only sufficient in providing a the-
oretical starting point and other means of identification are nec-
essary. Typically, a number of options are available. Selection of
the most appropriate technique/s depends on the stakeholders in-
volved in the grid development and the resources available to the
developer or development team. Process areas may be solicited
by interviewing a panel of experts, synthesizing the most critical
and most frequently mentioned concepts in literature, and/or a
combination of the two in either order [71]. Alternatively, un-
derstanding and recognizing organizational process goals may
be a point of departure for defining the key processes [61]. This
could proceed as follows: first, defining associated goals, which
are considered necessary to achieve the organization’s overall
objective. Then, from the goals, key process areas can be de-
rived. For example, one could break safety management down
into safety demonstration, safety implementation, strategies re-
lating to sustaining companies’ capabilities in the long-term,
etc., and find processes associated with these categories that
show strategic and operational significance. The number of se-
lected process areas is dependent on the subject chosen, and
thus, also exhibits great variation. For reasons of feasibility and
logistics, an appropriate number of items for such an assessment
method is about 20 [72] (see also examples in the review sample
in Table I).
2) Select Maturity Levels (Rating Scale): The next step in
the development of a maturity grid is to define a set of matu-
rity levels. Different assessment frameworks use different rating
scales. For example, ISO 9001 measures compliance with all
requirements on a binary pass/fail scale (with scope for minor
nonconformance). The Baldrige Award offers points for each
implemented requirement. The software CMM, having been in-
spired by Crosby’s maturity grid and motivated by the lack of
formalization in measurement, introduces the concept of orga-
nizational maturity as an ordinal scale for measuring capability.
An explicit statement of the underlying rationale and consistent
implementation of a maturity grid is required to provide theo-
retical rigor. Levels need to be distinct, well defined, and need
to show a logical progression as clear definition eases interpre-
tation of results.
Deciding on what rationale informs the rating scale essentially
means deciding on a leverage point for organizational change.
Referring to the review on existing maturity grids in Section IV,
we discern different underlying notions, namely:
1) existence and adherence to a structured process (e.g., in-
frastructure, transparency, and formality);
2) alteration of organizational structure (e.g., job roles and
policy);
3) emphasis on people (e.g., skills, training, and building
relationships);
4) emphasis on learning (e.g., awareness, mindset, and atti-
tude).
In some examples, we find a mixture (see review in
Section IV).
3) Formulate Cell Text (Intersection of Process Areas and
Maturity Levels): Identification and formulation of behavioral
characteristics for capabilities or processes is one of the most im-
portant steps in developing a maturity grid assessment. Process
characteristics need to be described at each level of maturity.
To discriminate between levels, descriptions should be precise,
concise, and clear. This requires: 1) a decision on whether the
cell text is “prescriptive” or “descriptive”; 2) a justification of
the information source; and 3) a decision on the mechanism of
formulating the text descriptions.
a) Prescriptive or descriptive: In a prescriptive approach,
specific and detailed courses of action are suggested for each
maturity level of a process area. For a descriptive approach, the
focus is on a detailed account of the individual case and concerns
for direct comparability of results between application cases are
less paramount. The choice also has an impact on maintenance,
since detailed activities, if not sufficiently generic, need to be
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
maintained for relevance and accuracy. In summary, there are
a number of aspects to consider, e.g., the underlying rationale,
characteristics, and knowledge of the subject area.
The subject area might be technical or social. Prescribing de-
tailed activities of what should be done at what stage is easier
for technical issues. For example, if deciding on energy man-
agement, specific regulations can be used, whereas for a social
issue, such as teamwork, a generally acceptable and widely ap-
plicable detailed prescription might be more difficult.
In addition to the consideration of whether a subject area is
more technical or more social in nature, DeBruin and Rosemann
[14] point to another consideration, which asks whether a field is
well established or new. Given the answer, different actions can
be taken to define individual cell descriptions and even maturity
levels. Actions include, for example, a top-down or a bottom-up
approach [14]. In a top-down approach, definitions are written
first, and then, measures or a set of practices are developed to fit
the definitions. In a bottom-up approach, the requirements and
measures are determined first, and then, definitions are writ-
ten to reflect these. A top-down approach works well if the
field is relatively new and there is little evidence of what is
thought to represent maturity. The emphasis, in this instance,
is first on what represents maturity, and then, how this can be
measured.
b) Information source: A number of options are available
to formulate the text descriptions in each cell: 1) by synthesizing
viewpoints from a sample representing the future recipients of
the assessment; or 2) by reviewing and comparing practices
of a number of organizations, for example, through empirical
studies, reviewing written case studies in literature, and best
practice guides from excellence initiatives.
c) Formulation mechanism: There are two mechanisms
to formulate the actual cell text as rating scale. One is to iden-
tify extreme ends of the scale, i.e., best and worst practices,
and then, to determine characteristics of all the stages in be-
tween. In this case, key tasks and procedures considered to
represent best practice should be based on discussion with rele-
vant stakeholders and experts in the field. This strategy assumes
that the rationale for individual cell descriptions is inductively
generated from the descriptions of practices. Alternatively, in-
dividual text descriptions for the cells in each selected process
area to be assessed are deduced from the underlying rationale
and formulated accordingly. However, this depends on the de-
cision as whether a definition is prescriptive or descriptive in
nature.
4) Define Administration Mechanism: The administration
mechanism of a maturity grid is integral to the success of the
assessment. In choosing a mechanism, consideration needs to be
given to the aim of the assessment, and the resources and support
infrastructure available for conducting the assessment. This de-
cision point is important because different delivery mechanisms
emphasize different aspects. In reviewing extant approaches, the
choice of delivery method appears to be connected to the general
goals and objectives of the assessment. Approaches aiming at
raising awareness and improving performance that way appear
to select paper-based distribution mechanisms, be it through in-
terview and/or group workshops [62], [67], [73]. Approaches
151
aiming at benchmarking seem to prefer electronic distribution
systems for questionnaires to reach a wide variety and large
number of participants [11], [20]. A combination of the two is
possible.
a) Focus on process (raising awareness): Individual
scores are taken as prompts for a discussion and identification
of steps for improvement. A facilitated workshop can enable
participants to discuss what influenced their judgment in scor-
ing a process area and why might it deviate from his or her
colleagues’ perception. Overall, the emphasis lies on the (dis-
cursive) process of arriving at the result.
Completion of the grid in a group-administered work-
shop [67], [71], [74] has a number of advantages. The re-
sponse rate is high and single-respondent bias can be avoided.
Further, if respondents are unclear about the meaning of a
term, they can ask for clarification. This ensures participants
have a common reference point, which facilitates interpre-
tation of the resulting scores. As each process area on the
grid is addressed in a group, the workshop also functions
as a team-building exercise [69]. Finally, face-to-face interac-
tion in groups elicits a higher level of iterative engagement
with the subject area than a questionnaire that is completed
individually.
b) Focus on end results (benchmarking): Scores are col-
lated to give an overall assessment of the capability and an
overall maturity level for the project, business unit, company,
or any other chosen improvement entity. An overall assessment
assumes that all processes are of equal importance. However, as
individual scores for each process are averaged out, aggregation
of results can mask potentially outstanding performance in one
area or potentially weak performance in another. It also obscures
differences in individual scores, which are often interesting in-
tervention points. Overall, when benchmarking, emphasis lies
on the end result.
In either case, the main value of a maturity grid-based assess-
ment is to capture a companies’ own evaluation of the situation.
Participants’ scores might be used as motivational driver for
management to change the maturity level of their team, project,
or organization and organizational capabilities.
C. Phase III—Evaluation
The transition from Phase II to III is fluid. Grids are likely
to evolve over time, where, through continued use, difficulties
or limitations may be revealed [61], [67]. As the assessment is
used and feedback gained from the experience of companies,
the grid should be iteratively refined. Evaluations may be con-
tinued until a saturation point is reached, i.e., until no more
significant changes are being suggested by participants and/or
until evaluation results are satisfactory. The first applications
of the assessment should ideally be treated as a final stage of
evaluation.
Thus, Phase III—evaluation, is an important stage in the de-
velopment of a maturity grid and serves a number of functions.
For example, tests are used to validate the grid, to obtain feed-
back on whether the grid fulfilled the requirements when ap-
plied in practice, and to identify items for refinement. Ideally,
152 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
evaluations are conducted within companies or institutions that
are independent of the development. During this phase, it is im-
portant to test input into the grid (choices made during Phases
I and II) for validity and the results acquired by applying the
grid in practice for correctness—in case of benchmarking also
for generalizability.
1) Validation: Once a grid is populated, it must be tested
for validity and relevance. This includes checking whether good
translations of the constructs have been achieved. In other words,
evidence needs to be given for correspondence between the au-
thor’s findings and the understandings of participants of the
assessments. Acknowledging its difficulty and allowing for an
element of subjectivity, the maturity grid needs to be tested
for breadth of the domain covered. This requires a degree of
agreement on what particular elements need to be included or
excluded, justifying the use of the theoretical framework under-
lying the selection of process areas (Phase II).
In addition to testing the content of the grid for validity, it is
necessary to ensure that the results obtained through applying
the grid “in the field” are correct, accurate, and repeatable. A
case study approach to method evaluation may be employed.
Although case studies cannot provide the scientific rigor of for-
mal experiments, they can provide sufficient information to help
judge if specific methods will benefit a project or an organiza-
tion [76]. If benchmarking is desired, results acquired through
the assessment need to be tested for generalizability.
2) Verification: In terms of verification, through application,
the method developed needs to be evaluated against the success
criteria and requirements defined during Phase I—planning.
D. Phase IV—Maintenance
Phase IV, the maintenance phase, is an ongoing phase. Con-
tinued accuracy and relevance of a maturity grid will be ensured
by maintaining it over time. Access and provision of neces-
sary resources to maintain the grid will affect its evolution and
use. Maintenance becomes necessary as domain knowledge and
understanding broadens and deepens. Similarly, current best
practice becomes outdated as a result of, for example, new tech-
nological developments. Maintenance is especially necessary if
detailed and prescriptive activities have been specified in the
cell text. For example, if a certain limit for CO2 emissions was
specified in the grid, current regulation might require a differ-
ent limit, and thus, mandate adjustment. Documentation and
training material is required, especially if the facilitator of an
assessment is not the author.
In addition, if the tool was developed for benchmarking pur-
poses, it is particularly important to ensure accurate data storage
and retrieval. In general, while regular maintenance is recom-
mended, updating a tool to reflect current best practice can
compromise validity. Hence, if substantial changes are made
after a tool has been formally evaluated, the evaluation phase
needs to be repeated.
Finally, maintenance includes adequate documentation and
appropriate communication of the design process and results to
the academic community and findings from application cases to
(participating) practitioners.
E. Summary of All Phases and Decision Points
Decision points during all phases are summarized in Table II.
If desired, each decision point allows for participative involve-
ment of the user. Having described the roadmap’s four phases
and their respective decision points and options, Section VI
introduces an illustrative example of its implementation in
practice.
VI. EXPERIENCE IMPLEMENTING THE ROADMAP
Spanning four years of development and application, we
devised a maturity grid to assess communication manage-
ment in engineering design [62], [71] and named it the com-
munication grid method (CGM). Development of the ma-
turity grid progressed by alternating between cross-domain
literature reviews and empirical field studies in eight com-
panies. Three exploratory studies were conducted during
Phase I (planning) and Phase II (development), and five appli-
cations were undertaken during Phases II and III (evaluation).
In what follows, the development process is described in de-
tail, following the phases and decision points of the roadmap
suggested in the previous section of this paper.
A. Phase I—Planning
Exploratory studies on communication in new product devel-
opment were conducted in three companies in the aerospace, IT,
and engineering tools sectors. This comprised a total of eight
weeks of observation and 63 semistructured face-to-face inter-
views with junior engineers and senior personnel.
1) Specify Audience: Interview results showed that mis-
alignment of perceptions on factors influencing communication
occurred predominantly at team interfaces. Therefore, the as-
sessment tool needed to aim mainly at communication across
one interface. The improvement entity was therefore to be com-
munication across the specified team or department interface,
the results were to be aimed at the same people providing the
scores.
2) Define Aim: Human communication in complex product
development projects sees manifold configurations of people,
products, and processes, which makes each communication situ-
ation different. Best practice benchmarks or generally applicable
indices for communication, against which a particular situation
can be compared, are perhaps impossible to find. Therefore,
the assessment purpose chosen was to raise awareness amongst
participants and diagnose improvement opportunities for their
current practices.
3) Clarify Scope: The primary goal was to be domain-
specific, i.e., the assessment was predominantly aimed at com-
munication within engineering firms. The focus on factors in-
fluencing communication at a team or departmental interface
rather than centered on a particular product development phase
also enabled cross-domain assessment.
4) Define Success Criteria: A number of success criteria in
the form of high-level and specific requirements were defined
of which a few are mentioned in what follows. In general, the
maturity grid was to be usable by and useful for industry. Success
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS 153

TABLE II
DECISION POINTS AND DECISION OPTIONS FOR THE DEVELOPMENT PHASES OF A MATURITY GRID

criteria for usability pertained to clarity of language used and
ease of scoring and analyses functions provided. Usefulness was
determined by the tool’s ability to trigger reflection and learning
episodes among the participants of the maturity grid assessment.
As an example for a high-level requirement that emerged
through interview, we would like to mention different concep-
tualizations of communication. Design engineers and engineer-
ing managers were asked to describe their daily design tasks
in general and their interactions with colleagues in particular.
Descriptions and explanations of their work yielded a number
of communication problems. Descriptions suggest that intervie-
wees focused in their personal conceptualization of communi-
cation problems on different aspects. Some focused on the in-
formation to be selected, some focused on the act of information
transmission (the interaction or utterance), some highlighted un-
derstanding of the situation as a key aspect, and some mentioned
a combination of the above. These aspects are complementary.
Thus, the maturity grid method needed to incorporate many as-
pects of communication and raise awareness of precisely those.
This is important as foci (subconsciously) chosen determine the
solution space.
As an example for a specific requirement, during interview,
engineers mentioned a myriad of factors, ranging from indi-
vidual capabilities and information handling skills to differ-
ences in format of information representation, mutual trust,
roles and responsibilities, decision making, handling of tech-
nical conflicts, hierarchies, overview of sequence of tasks, and
terminology. Many engineers contended that it would be dif-
ficult to intervene as they were uncertain, which factor to
start looking at first. Consequently, the maturity grid assess-
ment needed to provide multiple entry points for organizational
change.
154 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
B. Phase II—Development
The roadmap reminds the developers of maturity grids to the-
oretically ground their selection of process areas and maturity
levels, which influences the formulation of the cell text and
ultimately forms the basis for assessment. The rationale under-
lying the selection of process areas for the CGM is Luhmann’s
system-evolutionary communication theory and the selection of
the maturity levels is grounded in Argyris’ and Schön’s learning
theory. Details are described in what follows.
1) Select Process Areas: The predominant objective was
to assess design communication within engineering firms. Al-
though communication is one of the oldest fields of human
inquiry—starting out with Aristotle’s rhetoric [77]—there is no
canon of theory to which all scientists refer. Literature on com-
munication theory is rich yet disparate with little cross referenc-
ing. In addition, if one looks at review articles or textbooks, ord-
ering criteria are different. For example, Littlejohn [78] grouped
contributions to communication theory according to their disci-
plinary origins, Craig [79] identified streams of communication
research, such as “cybernetic” and “critical” to name a few, and
Miller [80] differentiates between communication theories that
focus on perspectives, processes, and contexts. Anderson [81]
analyzed the contents of seven communication theory textbooks
and identified over 200 distinct “theories” of which only 18 of
the 249 (7%) were included in more than three books.
This said and referring to an elicited high-level requirement
of addressing and assessing the comprehensiveness of commu-
nication (see previous section), informational, interactional, and
situational aspects cannot be viewed in isolation. A perspective
that integrates all aspects is the system-evolutionary perspective,
which advocates the generation, evolution, and reproduction of
social systems through communication. Engineering design is
considered as a social system [82], [83] and communication is
seen as being a constitutive part of a social system [79], [84].
The view of the structure generating function of communica-
tion has been proliferated through works of Krippendorff [85].
The perspective of system generation through communication
finds a vocal proponent in the late sociologist Luhmann [84].
Communication, encompassing the three aspects information,
utterance, and understanding, constitutes an emergent property
of the interaction between many (at least two) individuals and
is characterized in this view as coordination of behavior.
A number of factors influence this coordination of behavior.
Inductive “coding” of interview data (see Phase I) yielded more
than 40 factors influencing communication. Factors included,
for example, individual capabilities and information handling
skills to differences in format of information representation,
mutual trust, roles and responsibilities, decision making, han-
dling of technical conflicts, hierarchies, overview of sequence of
tasks, and terminology. They were grouped under the headings
“product,” “information,” “individual,” “team,” and “organiza-
tion” [75], and subsequently, mapped against Luhmann’s three
aspects of communication. The resulting framework presents
a comprehensive, arguably, not exhaustive, and selection of
factors that forms input to the audit method. Factors are seen
as levers for change, and thus, form the process areas to be
assessed.
The roadmap suggests using more or less 20 process areas in
order for it to be a feasible assessment. Therefore, it was decided
to use a modular system and have a base of about 40 process
areas of which a selection would be used in negotiation with the
users.
2) Select Maturity Levels: From a system-theoretic perspec-
tive, it is particularly important to choose an approach that trig-
gers internal processes within the system under assessment.
Change can be initiated, yet not fully controlled. Change initia-
tives, however, are only processable if they are phrased in the
“language of the system” [84]. Social systems develop their own
perceptions of reality, which they use to orientate and position
themselves in relation to their environment. This perception of
reality creates the system’s knowledge base, which, at the same
time, functions as indicator and benchmark for its own behavior.
Learning is the act of developing this knowledge base further. It
occurs on both individual and organizational levels. In the most
successful cases, learning leads to a coherent reality construc-
tion of a system, and thus, the system development keeps up
with changes in the environment [86].
Therefore, the maturity levels were based on an adaptation
of the three learning types described by Argyris and Schön
[63], [64]. In addition to these three learning types, an extra,
preliminary stage was added.
1) Level A: It describes situations, where participants do not
engage with the questions asked in the grid, i.e., they do
not reflect on the connection between the factor in question
and communication.
2) Level B: It corresponds with the first type of learning
described by Argyris and Schön, and points out that some
action is changed in order to correct a mistake, but other
than that tasks are carried out as usual.
3) Level C: It matches Argyris’ and Schön’s second type of
learning: people at this stage, modify their actions, as well
as thinking critically about existing norms, procedures,
policies, and objectives that govern their actions. This
means that a mistake is not just corrected once, but that
the general situation that led to a mistake is taken into
consideration.
4) Level D: It corresponds to the third type of learning and
signifies a stage, where employees are aware of the influ-
ence of a given factor on communication and continuously
check whether the way things are handled or setup is still
appropriate for the given situation. At this stage, learn-
ing does not only happen when a mistake needs to be
corrected: participants have the general mindset of con-
tinuously adjusting and improving the situation.
Given the complexity of communication and the unique na-
ture of each project and each company, it is difficult to specify
in detail what behavior every team should adopt. Therefore, it
should be noted that stage “D” is not necessarily the best. Par-
ticipants decide what is most adequate for the given project and
resources available.
In summary, the concept of learning permeates the assess-
ment approach put forward here in two ways: first, individual
participants in the assessment will learn about their colleagues’
perceptions by going through the assessment exercise. Second,
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
maturity levels in the assessment follow the underlying rationale
of specific learning types.
3) Formulate Cell Text: Once the factors that influence the
communication process and levels of maturity assigned to each
factor were identified, matrices of cells (scorecards or grids)
were devised. In this case, a set of about 10 scorecards resulted.
An example of one factor is shown in Fig. 6. Aiming to use the
right language, the authors first suggested the cell descriptions,
and subsequently, elicited feedback from users until no more
hesitations of people scoring occurred. This was a laborious
process of iterative refinement.
4) Choose Administration Mechanism: Our main emphasis
was to enable learning by following the assessment. Due to
the tradeoff between time and cost constraints and wanting to
situate numerical scores within the context of occurrence, we
experimented with a number of administration mechanisms.
In the end, individual 30-min interviews were combined with
a 2-h group workshop in which the result of the completed
scorecards, causes and solutions for identified problem areas
and gaps between the current and desired states were discussed.
C. Phase III—Evaluation
The CGM was applied to the following interfaces:
1) software and service design developing middleware (Ap-
plication Case 1: information technology);
2) the management team of a small engineering company
(Application Case 2: engineering tools manufacturer);
3) design and service of an aero engine (Application Case 3:
aerospace manufacturer defense);
4) embodiment design and simulation of the design of an
automotive vehicle series (Application Case 4: automotive
manufacturer);
5) preliminary design and detail design of an aero engine tur-
bine blade (Application Case 5: aerospace manufacturer
civil).
Development and evaluation of the CGM evolved through
rounds of iteration between each application case. We used
member validation [38] to obtain feedback on the outcome of the
assessment (reliability of results) as well as, for example, func-
tionality and usability of the method. After each application, the
results were fed back to the participants of the studies through
workshops, a final presentation, and in some cases, a written
report. Repeatedly, participants felt the results accurately rep-
resented the actual status of communication between the team
interfaces chosen. This ensured reliability. The method itself
was evaluated according to set criteria, such as functionality
and usability, usefulness and learn effect, triggering reflection,
and correctness of results obtained [68], [69]. Findings from
verbal feedback were cross-referenced with a questionnaire us-
ing the same criteria and completed by the same participants.
For a comprehensive description refer to Maier [75]. In addition
to feedback from experts in industry participating in the studies,
feedback was also sought from experts from two engineering
consultancies and engineering design researchers from a variety
of universities. The different respondent groups were chosen be-
cause a single group of respondents could not properly replicate
155
broader use in industry or judge scientific purpose. As research
progressed and predominantly during the three early applica-
tion cases after which the design was frozen, subtle changes
were made to the list of process areas, the number of interfaces
assessed at any one time, and the administration mechanism.
Description of the development process reflects the final design
of the maturity grid.
D. Phase IV—Maintenance
The CGM aims to raise awareness. It may compare teams
and departments within companies with each other, rather than
positing a best practice benchmark. Cell text is descriptive, and
therefore, the maturity grid will remain up-to-date. The develop-
ment process was documented and communicated to scientific
audiences through peer-reviewed publications. Documentation
of results from the application cases were made available to
participating industry partners.
Following the roadmap in the development of the communica-
tion grid gave us a clear structure for development. It also meant
that both the authors and the users of the assessment method
knew what we were expecting and could therefore define the
rules of engagement and estimate time and effort.
In keeping with the design science perspective underlying
this paper (see Section II), a further evaluation of the CGM and
the suggested roadmap for development of such maturity as-
sessment methods was undertaken (see Table III). We followed
the guidelines for devising an artifact formulated by Hevner
et al. [36] and reformulated and used specifically in the context
of requirements for maturity models by Becker et al. [12].
VII. SUMMARY AND CONCLUSION
This paper has discussed various notions of maturity, i.e.,
“process maturity,” “organizational maturity,” “process capabil-
ity,” “project maturity,” and “maturity of organizational capabil-
ities” (see Section III). It presented a comprehensive overview
of 24 extant maturity grids that build on the ideas of Crosby’s
QMMG from the late 1970s (see Section IV). In reviewing,
particular emphasis was placed on analyzing embedded as-
sumptions about organizational change in the maturity scales of
the examples reviewed. As direct comparison, leverage points
for maturing collaboration as an organizational capability were
shown to be one or a combination of: existence and adherence
to a structured process (e.g., infrastructure, transparency, and
formality), alteration of organizational structure (e.g., job roles
and policy), emphasis on people (e.g., skills, training, and build-
ing relationships), and/or emphasis on learning (e.g., awareness,
mindset, and attitude). This shows that combining different per-
spectives and measures of “goodness” for one and the same
process or capability is difficult. Assessing maturity will, there-
fore, perhaps always be more subjective than objective [47].
While the number of maturity grids and models is grow-
ing, there is little support available on how to develop these
approaches to organizational capability assessment and devel-
opment. To address this issue, the paper also provided a four-
phase roadmap for developing maturity grids (see Section V)
and showed the roadmap’s implementation in industry with an
156 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012

TABLE III
EVALUATION OF THE ROADMAP TO DEVELOP MATURITY GRIDS AND THE CGM

illustrative example (see Section VI). This roadmap is a first at-
tempt at identifying and synthesizing phases and decision points
that may be useful to both authors of assessment grids and im-
plementers, who need to handle the multimodel environment.
It provides the parameters within which professional develop-
ment of a maturity grid might occur. Further, it provides the
parameters within which professional judgments for evaluation
of existing grids might be made. However, it cannot, and does
not aim to provide the answer to every dilemma an author of a
grid may face. Decision points and options provide instances for
reflection to ensure appropriate courses of action when devel-
oping new or evaluating existing maturity grids. Such instances
occur, for example, when selecting process areas and facing the
issue of academic rigor versus logistical feasibility, and thereby,
practical utility. For academic purposes, the list of process areas
chosen needs to be comprehensive, complete, correct, consistent
and, above all, theoretically justified. For industrial applicabil-
ity, however, certain flexibility for adaptation and tailoring needs
to be designed into the method. It is necessary to strike a balance
between developing an exhaustive method and a usable one.
A. Implications for Industrial Practice
Maturity grids are built upon conceptual models that in their
own rights provide insights into the author’s perspective of
the factors important in an organization. Thus, the maturity
grid-based assessment methods collectively offer a contempo-
rary representation of different conceptualizations of organiza-
tional practices and capabilities that are viewed as important
for success. In addition, this review presents an overview of the
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
different maturity grid approaches available to assess organiza-
tional capabilities and initiate change processes at a given point
in time. This provides organizations with a better understand-
ing of existing capabilities and enables benchmarking against a
range of competitors. While maturity grids may share a common
structure, their content differs and very often they are developed
anew. This paper provides a common reference point and guid-
ance for the evaluation of existing grids and the development of
new grids. It alerts both the novice and expert user to the poten-
tial decisions to be addressed at the start of the development of
a maturity grid assessment tool.
B. Implications for Research
The maturity grids selected all embrace the notion that suc-
cessful organizational change can be triggered and/or achieved
by an assessment of practices. Most maturity grids are based
around the underpinning assumption that key organizational ca-
pabilities need to be rooted in codified business processes. Thus,
there is an acceptance that business processes are beneficial and
necessary. However, the underlying idea of “cause and effect”
might be fallacious because the processes under assessment
are in many cases social processes that do not follow simple
cause and effect patterns [87]. Careful analysis is necessary,
however, to discern whether maturity grid methods fall into a
naturalistic–mechanistic perspective that holds processes to be
fully quantifiable and controllable. Given the variety of maturity
grids available, the skeptical observer would wisely enquire as
to the basis upon which each maturity grid is founded. A num-
ber of perspectives on organizational change—certain ideas of
goodness based on certain paradigms of goodness—are em-
bedded in the rating scales of maturity grids. The overview,
analysis, and suggestions provided here gives a background for
research into theory building on the development of maturity
grids and for research into management tools as interventions
in organizations.
ACKNOWLEDGMENT
The authors would like to thank the constructive comments
provided by the three anonymous reviewers, the Associate Edi-
tor Prof. J. Pinto, and the Editors-in-Chief, Profs. G. Farris and
R. Sabherwal. They would also like to thank the following peo-
ple, who were contacted during the writing of this paper: Dr. B.
Bater, InfoPlex Associates, Prof. I. Cooper, Eclipse Consultants,
Dr. N. Crilly, University of Cambridge, Prof. K. Grant, Univer-
sity of Texas at San Antonio, Dr. M. Langen, Siemens AG, and
Dr. S. Macmillan, University of Cambridge. They also thank
the members of the Engineering Design Centre, University of
Cambridge, the members of the Design Management Group,
University of Cambridge, and the members of the section Work,
Technology, and Organization, Technical University of Den-
mark, who supported with helpful advice on earlier versions of
the manuscript.
REFERENCES
[1] M. C. Paulk, “Surviving the quagmire of process models, integrated
models, and standards,” presented at the ASQ Annu. Qual. Congr.,
Toronto, Canada, 2004.
157
[2] D. Leonard-Barton, “Core capabilities and core rigidities: A paradox in
managing new product development,” Strateg. Manage. J., vol. 13, no. 8,
pp. 111–125, 1992.
[3] J. W. Moore, “An integrated collection of software engineering stan-
dards,” IEEE Softw., vol. 16, no. 6, pp. 51–57, Nov./Dec. 1999.
[4] C. P. Halvorsen and R. Conradi, “A taxonomy to compare SPI frame-
works,” presented at the 8th Eur. Workshop Softw. Process Technol.,
Witten, Germany, 2001.
[5] M. C. Paulk, “A taxonomy for improvement frameworks,” presented at
the World Congr. Softw. Qual., Bethesda, MD, 2008.
[6] M. C. Paulk, B. Curtis, M. B. Chrissis, and C. V. Weber, “Capability
maturity model SM for software, version 1.1,” Carnegie Mellon Univ.,
Pittsburgh, PA, Tech. Rep. CMU/SEI-93-TR-024 ESC-TR-93-177, Feb.
1993.
[7] M. C. Paulk, C. V. Weber, B. Curtis, and M. B. Chrissis, The Capa-
bility Maturity Model: Guidelines for Improving the Software Process.
Boston, MA: Addison-Wesley, 1995.
[8] M. B. Chrissis, M. Konrad, and S. Shrum, CMMI. Guidelines for Process
Integration and Product Improvement. Boston, MA: Addison-Wesley,
2003.
[9] B. Curtis, W. E. Hefley, and S. Miller, The People Capability Matu-
rity Model: Guidelines for Improving the Workforce. Reading, MA:
Addison Wesley Longman, 2002.
[10] M. Hammer, “The process audit,” Harv. Bus. Rev., vol. 85, no. 4, pp. 111–
121, 2007.
[11] K. P. Grant and J. S. Pennypacker, “Project management maturity: An
assessment of project management capabilities among and between se-
lected industries,” IEEE Trans. Eng. Manage., vol. 53, no. 1, pp. 59–68,
Feb. 2006.
[12] J. Becker, R. Knackstedt, and J. Pöppelbuß, “Developing maturity models
for IT management: A procedure model and its application,” Bus. Inf.
Syst. Eng., vol. 1, no. 3, pp. 213–222, 2009.
[13] M. Kohlegger, R. Maier, and S. Thalmann, “Understanding maturity
models results of a structured content analysis,” presented at the I-
KNOW I-SEMANTICS, Graz, Austria, 2009.
[14] T. DeBruin and M. Rosemann, “Understanding the main phases of de-
veloping a maturity model,” presented at the 16th Australas. Conf. Inf.
Syst., Sydney, NSW Australia, 2005.
[15] M. Siponen, “Towards maturity of information security maturity crite-
ria: Six lessons learned from software maturity criteria,” Inf. Manage.
Comput. Secur., vol. 10, no. 5, pp. 210–224, 2002.
[16] L. G. Pee and A. Kankanhalli, “A model of organisational knowledge
management maturity based on people, process, and technology,” J. Inf.
Knowl. Manage., vol. 8, no. 2, pp. 79–99, 2009.
[17] C. Ibbs and Y. Kwak, “Assessing project management maturity,” Proj.
Manage. J., vol. 31, no. 1, pp. 32–43, 2000.
[18] J. S. Pennypacker and K. P. Grant, “Project management maturity: An
industry benchmark,” Proj. Manage. J., vol. 34, no. 1, pp. 4–11, 2003.
[19] M. Mullaly, “Longitudinal analysis of project management maturity,”
Proj. Manage. J., vol. 36, no. 3, pp. 62–73, 2006.
[20] U. Kulkarni and R. St Louis, “Organisational self assessment of knowl-
edge management maturity,” presented at the 9th Americas Conf. Inf.
Syst., Pittsburgh, PA, 2003.
[21] T. Mettler and P. Rohner, “A design science research perspective on
maturity models in information systems,” University of St. Gallen, St.
Gallen, May 2009.
[22] R. Phaal, C. J. P. Farrukh, and D. R. Probert, “Technology management
tools: Concept, development and application,” Technovation, vol. 26,
no. 3, pp. 336–344, 2005.
[23] D. O. Health. (2010, Sep 6). Achieving Excellence Design Evaluation
Toolkit (AEDET Evolution) last accessed [Online]. Available: http://
www.dh.gov.uk/en/Publicationsandstatistics/Publications/Publications
PolicyAndGuidance/DH_082089.
[24] V. P. Kochikar, “The knowledge management maturity model—A staged
framework for leveraging knowledge,” presented at the KMWorld 2000,
San Jose, CA, 2000.
[25] K. Ehms and M. Langen, “Reifemodelle im KM-consulting—
Erfahrungen aus 4 Jahren Beratungspraxis,” presented at the KnowTech
2004, Muenchen, Germany, 2004.
[26] P. Chinowsky, K. Molenaar, and A. Realph, “Learning organizations in
construction,” J. Manage. Eng., vol. 23, no. 1, pp. 27–33, 2007.
[27] J. C. Mankins, “Technology readiness levels: A white paper,” presented
at the NASA, Washington, DC, 1995.
[28] J. C. Mankins, “Technology readiness assessments: A retrospective,”
Acta Astronautica, vol. 65, no. 9–10, pp. 1216–1223, 2009.
158 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 59, NO. 1, FEBRUARY 2012
[29] J. E. Ramirez-Marquez and B. Sauser, “System development planning
via system maturity optimization,” IEEE Trans. Eng. Manage., vol. 56,
no. 3, pp. 533–548, Aug. 2009.
[30] R. Foster, Innovation: The Attacker’s Advantage. New York: Summit
Books, 1986.
[31] C. M. Christensen, “Exploring the limits of the technology S-Curve.
Part I: Component technologies,” Prod. Oper. Manage., vol. 1, no. 4,
pp. 334–357, 1992.
[32] R. Nolan, “Managing the computer resource: A stage hypothesis,” Harv.
Bus. Rev., vol. 16, no. 7, pp. 399–405, 1973.
[33] L. E. Greiner, “Evolution and revolution as organizations grow,” Harv.
Bus. Rev., vol. 50, no. 4, pp. 37–46, 1972.
[34] N. Cross, Engineering Design Methods: Strategies for Product Design.
London, U.K.: Wiley, 2008.
[35] N. Cross, “Designerly ways of knowing: Design discipline versus design
science,” Des. Issu., vol. 17, no. 3, pp. 49–55, 2001.
[36] A. R. Hevner, S. R. March, J. Park, and S. Ram, “Design science in
information systems research,” MIS Q., vol. 28, no. 1, pp. 75–105, 2004.
[37] K. M. Eisenhardt, “Building theories from case study research,” Acad.
Manage. Rev., vol. 14, no. 4, pp. 532–550, 1989.
[38] M. Bloor, “Techniques of validation in qualitative research: A critical
commentary,” in Context and Method in Qualitative Research, G. Miller
and R. Dingwall, Eds. London, U.K.: Sage, 1997, pp. 37–50.
[39] Oxford English Dictionary (OED) “Mature” and “Maturity,” 2009.
[40] K. Dooley, A. Subra, and J. Anderson, “Maturity and its impact on new
product development project performance,” Res. Eng. Des., vol. 13,
no. 1, pp. 23–39, 2001.
[41] J. K. Crawford, Project Management Maturity Model Providing a Proven
Path to Project Management Excellence. New York: Marcel Dekker,
2002.
[42] J. K. Crawford, Project Management Maturity Model, 2nd ed. Boca
Raton, FL: Auerbach Publications, Taylor and Francis Group, 2007.
[43] Project Management Institute, A Guide to the Project Management Body
of Knowledge. Newtown Square, PA: Project Management Institute,
2000.
[44] T. J. Cooke-Davies and A. Arzymanow, “The maturity of project man-
agement in different industries: An investigation into variations between
project management models,” Int. J. Proj. Manage., vol. 21, no. 6,
pp. 471–478, 2003.
[45] H. Kerzner, Using the Project Management Maturity Model: Strategic
Planning for Project Management, 2nd ed. Hoboken, NJ: Wiley, 2005.
[46] R. Gareis and M. Huemann, “Maturity models for the project-oriented
company,” in Gower Handbook of Project Management, J. R. Turner,
Ed., 4th ed. Aldershot, U.K.: Gower, 2008, pp. 183–208.
[47] E. S. Andersen and S. A. Jessen, “Project maturity in organisations,” Int.
J. Proj. Manage., vol. 21, no. 6, pp. 457–461, 2003.
[48] Y. H. Kwak and C. W. Ibbs, “Calculating project management’s return
on investment,” Proj. Manage. J., vol. 31, no. 2, pp. 38–47, 2000.
[49] G. Skulmoski, “Project maturity and competence interface,” Cost Eng.,
vol. 43, no. 6, pp. 11–18, 2001.
[50] Project Management Institute, Organizational Project Management Ma-
turity Model (OPM3). Newtown Square, PA: Project Management
Institute, 2005.
[51] E. G. Penrose, The Theory of the Growth of the Firm. New York:
Wiley, 1959.
[52] B. Wernerfelt, “A resource-based view of the firm,” Strateg. Manage.
J., vol. 5, no. 2, pp. 171–180, 1984.
[53] J. B. Barney, “Firm resources and sustained competitive advantage,” J.
Manage., vol. 17, no. 1, pp. 99–120, 1991.
[54] K. M. Eisenhardt and C. B. Schoonhoven, “Resource-based view of
strategic alliance formation: Strategic and social effects in entrepreneurial
firms,” Org. Sci., vol. 7, no. 2, pp. 136–150, 1996.
[55] D. Ulrich and N. Smallwood, Why the Bottom Line isn’t!: How to Build
Value Through People and Organizational Change. New York: Wiley,
2003.
[56] D. Ulrich and N. Smallwood, “Capitalizing on capabilities,” Harv. Bus.
Rev., vol. 82, no. 6, pp. 119–127, 2004.
[57] D. Teece, G. Pisano, and A. Shuen, “Dynamic capabilities and strategic
management,” Manage. J., vol. 18, no. 7, pp. 509–533, 1997.
[58] A. Fincher, “Project management maturity model,” presented at the 28th
Annu. Semin. Symp., Chicago, IL, 1997.
[59] R. Szakonyi, “Measuring R&D effectiveness—II,” Res.—Technol. Man-
age., vol. 37, pp. 44–55, May/Jun. 1994.
[60] Constructing Excellence, Effective Teamwork. A Best Practice Guide for
the Construction Industry. Watford, Herfordshire, U.K., 2004.
[61] J. E. Strutt, J. V. Sharp, E. Terry, and R. Miles, “Capability maturity
models for offshore organisational management,” Environ. Int., vol. 32,
no. 8, pp. 1094–1105, 2006.
[62] A. M. Maier, C. M. Eckert, and P. J. Clarkson, “Identifying requirements
for communication support: A maturity grid-inspired approach,” Expert
Syst. Appl., vol. 31, no. 4, pp. 663–672, 2006.
[63] C. Argyris and D. Schön, Organizational Learning: A Theory of Action
Perspective. Reading, MA: Addison-Wesley, 1978.
[64] C. Argyris and D. Schön, Organization Learning II: Theory, Method and
Practice. Reading, MA: Addison-Wesley, 1978b.
[65] K. E. Emam and D. R. Goldenson, “An empirical review of software
process assessments,” presented at the Nati. Res. Counc. Can., Inst. Inf.
Technol., Ottawa, ON, Nov. 1999.
[66] M. C. Paulk, S. Guha, W. E. Hefley, E. B. Hyder, and M. Iqbal, “Com-
paring the eSCM-SP and CMMI: A comparison between the e-sourcing
capability model for service providers v2 and the capability maturity
model integration v1.1,” Carnegie Mellon Univ., Pittsburgh, PA, IT Ser-
vices Qualification Center, Dec. 2005.
[67] J. Moultrie, “Development of a design audit tool for SMEs,” J. Prod.
Innov. Manage., vol. 24, no. 4, pp. 335–368, 2007.
[68] J. Wilson, “Communication artifacts. The design of objects and the object
of design,” in Design and the Social Sciences. Making Connections,
J. Frascara, Ed. London, U.K.: Taylor and Francis, 2002, pp. 24–
32.
[69] V. Chiesa, P. Coughlan, and C. Voss, “Development of a technical in-
novation audit,” J. Prod. Innov. Manage., vol. 13, no. 2, pp. 105–136,
1996.
[70] R. Szakonyi, “Measuring R&D effectiveness—I,” Res.—Technol. Man-
age., vol. 37, pp. 27–32, Mar./Apr. 1994.
[71] A. M. Maier, M. Kreimeyer, U. Lindemann, and J. P. Clarkson, “Reflect-
ing communication: A key factor for successful collaboration between
embodiment design and simulation,” J. Eng. Des., vol. 20, no. 3, pp. 265–
287, 2009.
[72] J. Moultrie, “Development of a design audit tool to assess product design
capability,” in Institute for Manufacturing. Cambridge, MA: Univ. of
Cambridge, 2004.
[73] P. Fraser, C. Farrukh, and M. Gregory, “Managing product development
collaborations—A process maturity approach,” J. Eng. Manufacture,
Proc. Instn Mech. Engrs, Part B, vol. 217, no. B11, pp. 1499–1519,
2003.
[74] P. Fraser, J. Moultrie, and M. Gregory, “The use of maturity models/grids
as a tool in assessing product development capability,” presented at the
IEEE Int. Eng. Manage. Conf., Cambridge, U.K., 2002.
[75] A. M. Maier, “A grid-based assessment method of communication in
engineering design,” in Engineering Department. Cambridge, U.K.:
Univ. of Cambridge, 2007.
[76] B. Kitchenham and S. L. Pfleeger, “Case studies for method and tool
evaluation,” IEEE Softw., vol. 12, no. 4, pp. 52–62, Jul. 1995.
[77] W. R. Roberts, “Rhetorica,” in The Works of Aristotle, vol. XI, W. D.
Ross, Ed. New York: Oxford Univ. Press, 1936.
[78] S. W. Littlejohn, Theories of Human Communication. Belmont, CA:
Wadsworth Publ., 1999.
[79] R. T. Craig, “Communication theory as a field,” Commun. Theory, vol. 9,
no. 2, pp. 119–161, 1999.
[80] K. Miller, Communication Theories: Perspectives, Processes, and Con-
texts, 2nd ed. New York: McGraw Hill, 2005.
[81] J. A. Anderson, Communication Theory. Epistemological Foundations.
New York: The Guilford Press, 1996.
[82] L. L. Bucciarelli, “Reflective practice in engineering design,” Des. Stud.,
vol. 5, no. 3, pp. 185–190, 1984.
[83] L. L. Bucciarelli, Designing Engineers. Cambridge, MA: MIT Press,
1994.
[84] N. Luhmann, Social Systems. Stanford, CA: Stanford Univ. Press,
1995.
[85] K. Krippendorff, “Communication and the genesis of structure,” Gen.
Syst., vol. 16, pp. 171–185, 1971.
[86] E. V. Glasersfeld, “Konstruktion der Wirklichkeit und der Begriff der
Objektivität,” in Einführung in Den Konstruktivismus, H. Gumin and
A. Mohler, Eds. München, Germany: München, 1985, pp. 1–26.
[87] S. H. Pfleeger, N. Fenton, and S. Page, “Evaluating software engineering
standards,” IEEE Comput., vol. 27, no. 9, pp. 71–79, Sep. 1994.
[88] P. B. Crosby, Quality is Free: The Art of Making Quality Certain. New
York: Penguin, 1979.
[89] A. M. Maier, M. Kreimeyer, C. Hepperle, C. M. Eckert, U. Linde-
mann, and P. J. Clarkson, “Exploration of correlations between factors
MAIER et al.: ASSESSING ORGANIZATIONAL CAPABILITIES: REVIEWING AND GUIDING THE DEVELOPMENT OF MATURITY GRIDS
influencing communication in complex product development,” Concur-
rent Eng. Res. Appl., vol. 16, no. 1, pp. 37–59, 2008.
[90] R. Hughes, “The safety management maturity grid,” in Proc. Prof. Saf.,
Jun. 1985, pp. 15–18.
[91] R. A. Radice, J. T. Harding, P. E. Munnis, and R. W. Philips, “A pro-
gramming process study,” IBM Syst. J., vol. 24, no. 2, pp. 91–101,
1985.
[92] Building Research Energy Conservation Support Unit (BRECSU), Re-
viewing Energy Management. Watford, U.K.: Building Research Es-
tablishment, Jan. 1993.
[93] J. Hackos, “The information process maturity model: A 2004 update,”
Best Pract., vol. 6, no. 4, pp. 1–8, 2004.
[94] M. E. McGrath and C. L. Akiyama, “PACE: An integrated process
for product and cycle-time excellence,” in Setting the PACE in Product
Development: A Guide to Product And Cycle-Time Excellence, M. E.
McGrath, Ed. revised edition, Boston, MA: Butterworth-Heinemann,
1996, pp. 17–30.
[95] T. R. Stacey, “The information security program maturity grid,” Inf.
Syst. Secur., vol. 5, pp. 22–33, 1996.
[96] D. A. Hillson, “Towards a risk maturity model,” Int. J. Proj. Bus. Risk
Manage., vol. 1, no. 1, pp. 35–45, 1997.
[97] S. Austin, A. Baldwin, J. Hammond, M. Murray, D. Root, D. Thomson,
and A. Thorpe, Design Chains—A Handbook for Integrated Collabora-
tive Design. London, U.K.: Thomas Telford, 2001.
[98] M. Bruce and J. Bessant, Design in Business. Strategic Innovation
Through Design. London, U.K.: Financial Times Press, 2002.
[99] D. M. Fisher, “The business process maturity model. A practical approach
to identifying opportunities for optimization,” in Proc. BP Trends, Sep.
2004, pp. 1–7.
[100] R. Woodall, I. Cooper, D. Crowhurst, M. Hadi, and S. Platt, “MaSC:
Managing sustainable companies,” Eng. Sustainability, vol. 157, no. 1,
pp. 15–21, 2004.
[101] K. B. Kahn, G. Barczak, and R. Moss, “Perspective: Establishing an
NPD best practices framework,” J. Prod. Innov. Manage., vol. 23, no. 2,
pp. 106–116, 2006.
[102] D. M. Ashcroft, C. Morecroft, D. Parker, and P. R. Noyce, “Safety culture
assessment in community pharmacy: Development, face validity, and
feasibility of the Manchester Patient Safety Assessment Framework,”
Qual. Saf. Health Care, vol. 14, no. 6, pp. 417–421, 2005.
[103] D. Parker, M. Lawrie, and P. Hudson, “A framework for understanding
the development of organisational safety culture,” Saf. Sci., vol. 44,
no. 6, pp. 551–562, 2006.
[104] D. Parker, M. Lawrie, J. Carthey, and M. Coultous, “The Manchester
patient safety framework: Sharing and learning,” Clin. Risk, vol. 14,
no. 4, pp. 140–142, 2008.
[105] M. van Steenbergen, R. Bos, S. Brinkkemper, I. van de Weerd, and
W. Bekkers, “The design of focus area maturity models,” in Global
Perspectives on Design Science Research. Lecture Notes in Computer
Science, R. Winter, J. L. Zhao, and S. Aier, Eds. Berlin, Heidelberg:
Springer-Verlag, 2010, pp. 317–332.
[106] M. Rohloff, “Case study and maturity model for business process man-
agement implementation,” in U. Dayal, J. Eder, J. Koehler, and H. Rei-
jers. Business Process Management. (Lecture Notes Comput. Sci.) eds.
Springer: Berlin/Heidelberg. vol. 5701, pp. 128–142, 2009.
[107] Z. Radnor, “Lean in public services: Is it a panacea or paradox?” Public
Exe. J., pp. 24-25, Jan./Feb. 2009.
159
Anja M. Maier received the M.A. degree in politi-
cal science, communication science, and philosophy
from the University of Münster, Germany and the
Ph.D. degree in engineering design from the Univer-
sity of Cambridge, Cambridge, U.K.
She is currently an Associate Professor in engi-
neering management with the Technical University of
Denmark, Lyngby, Denmark. She was also a Consul-
tant in the manufacturing and software industries. Her
research interests include the areas of design process
improvement, interface management, change man-
agement, capability assessment, collaborative designing, communication, and
innovation in design.
Prof. Maier is a member of the Academy of Management, the Design Society,
and a Fellow of the Cambridge Philosophical Society.
James Moultrie received the B.Tech (Hons) de-
gree in mechanical engineering from Lough-
borough University, U.K., the M.A. degree in
industrial design from DeMontfort University, Le-
icester, U.K., the M.B.A. degree from Loughborough
University, U.K., and the Ph.D. degree in engineer-
ing from the University of Cambridge, Cambridge,
U.K.
He is currently a Senior Lecturer in innovation
and design management with the University of Cam-
bridge, Cambridge, U.K. He is a Chartered Mechan-
ical Engineer (IMechE) and has as also been a Product Marketing Manager,
Project Manager, and Senior Engineer in the precision instruments sector. His
research interests include the areas of innovation and design management.
Dr. Moultrie was the recipient of a Scientific and Technical Academy Award
and an Emmy for work on a range of lenses for professional cinematography in
2000.
P. John Clarkson received the B.Eng., M.A.
(Cantab), and the Ph.D. degrees in engineering from
the University of Cambridge, Cambridge, U.K.
He is currently a Professor of Engineering Design
and the Director of the Engineering Design Centre
at the University of Cambridge, Cambridge, U.K. He
was also a PA Consultant, where he was engaged in
product development practice, with a focus on med-
ical equipment and projects for the U.K. Ministry of
Defence. He has authored or coauthored more than
400 papers in the past ten years and also a number of
practitioner workbooks on medical equipment design and inclusive design. His
research interests include the general area of engineering design, in particular,
the development of design methodologies to address specific design issues.
Prof. Clarkson is a Chartered Engineer, a Fellow of the Institution of Engi-
neering and Technology, and a member of the American Society of Mechanical
Engineers and the Design Society.