Documente Academic
Documente Profesional
Documente Cultură
86
86
Bases of Intra-Organizational Power: An Analysis of the Information
Technology Department
Andrew J. Setterstrom1 and J. Michael Pearson2
1
College of Business, Northern Illinois University, USA
2
College of Business, Southern Illinois University Carbondale, USA
asetterstrom@niu.edu
Abstract: Power is an important aspect of the social systems that make up organizations. The concept of power
helps to explain how organizational decisions are made and executed, despite opposition that results from
competing goals and desires amongst those in which a given decision affects. Using resource dependence and
strategic contingency theories for guidance, we identify potential sources of intra-organizational power for the
information technology (IT) department and its members. The comprehensive list of propositions developed in
this paper will provide researchers potential hypotheses to test in future research, as well as a means to assess
overall IT departmental power. While power plays an important role in facilitating the execution of the activities by
an organization’s IT department, it is often pursued by self-interested individuals, due to the fact that it provides
the ability to influence decisions, such as resource allocation, as well as providing a sense of control over
organizational outcomes and personal satisfaction. Based on the propositions developed in this paper, we
demonstrate how power considerations provide one plausible explanation for many of the poor organization
outcomes that occur with respect to the IT function, including technology for technology’s sake, a lack of user
preferences being integrated into IT systems development projects, resistance to using of knowledge
management systems, and resistance to IS outsourcing. Our discussion of power-gaining activities provides
practitioners an explanation of dysfunctional behaviors that previously may have been perceived as irrational, or
even undetected. In our discussion section, we provide suggestions for researching the propositions we have
developed. In particular, we suggest that a longitudinal or multi-case study approach may provide the best
method for researchers to test our set of propositions. At the same time, we caution that results from case studies
would be difficult to generalize, as the configuration of IT solutions adopted by organizations tend to be
idiosyncratic.
1. Introduction
The study of power in social systems has long been an important pursuit for scholars, in the context of
both intra- and inter-organizational situations. In particular, the past three decades have generated an
increased interest in power in the mainstream information systems (IS) research and has precipitated
considerable examination (see Jasperson et al., 2002). This is not surprising, as research has
suggested that power plays a critical role in execution of business functions carried out by an
organization’s IT department. For instance, IT departmental power has been demonstrated to be an
important aspect of IT governance, affecting the ability of the IT function to control the development of
IT projects, as well as to successfully propose IT investments for implementation (Xue, Liang, and
Boulton, 2008). In the same vein, Preston, Chen, and Leidner (2008) argue that IT managers,
specifically CIOs, need some degree of decision-making authority to pursue IT initiatives that support
organizational strategies. It has also been shown that individuals within organizations resist the
implementation of IT perceived to shift the distribution of power away from them (Alvarez, 2008;
Lapointe and Rivard, 2005; Sarker, Sarker, and Sidorova, 2006). Consequently, power considerations
for information systems implementation are important in avoiding implementation failure.
Prior scholarly investigations have attempted to gauge the power possessed by an organization’s IT
department by conducting empirical research into the perceived power relative to other departments
(e.g. Lucas, 1984; Lucas and Palley, 1987; Saunders and Scamell, 1986). While providing valuable
insights, these studies tend to be problematic in that perceptions often do not account for unenacted
power (Lucas, 1984; Pfeffer, 1981), while technology users may even fail to recognize when power is
being leveraged (Provan, 1980). Sillince and Mouakket (1997) observed that power is
multidimensional and that much of the IT research has historically adopted too narrow of an
operationalization; they argue that multiple views of power are necessary in gaining a more complete
understanding of power’s complexities. Pfeffer (1981) suggests one possible method for accurately
gauging power is to utilize Gamson’s (1968) indirect approach: assessment of the potential sources of
power. This indirect approach has the ability to detect power that is unutilized or which is exercised
without being perceived by other actors within the organization.
ISSN 1566-6379 88 ©Academic Publishing International Ltd
Reference this paper as: Andrew J. Setterstrom and J. Michael Pearson, “Bases of Intra-Organizational Power:
An Analysis of the Information Technology Department” The Electronic Journal Information Systems Evaluation
Volume 16 Issue 2 2013, (88-102) , available online at www.ejise.com
Andrew J. Setterstrom and J. Michael Pearson
The contribution of this paper is to provide a comprehensive examination of critical sources of power
for an organization’s IT department. Drawing from the theories of resource dependence and strategic
contingency as a theoretical framework (Hickson et al., 1971; Hinings et al., 1974; Pfeffer and
Salancik, 1978), a set of propositions are developed that identify potential sources of organizational
power for IT departments that are intended for use in future research. Information about the sources
of power for IT departments would provide both researchers and practitioners examining the decision-
making authority of the IT department an approach to assessing departmental power.
In addition to providing propositions, we demonstrate how one driving factor many common problems
organizations experience relating to their IT function can be traced to, in part, attempts to gain
legitimacy and power by IT personnel. The misuse of power by organizational leaders has been well
documented (see Higgs, 2009). Previous literature has suggested many possible sources for issues
relating to an organization’s IT function, such as a low level of shared understanding between the CIO
and top management about the role of IT within an organization (Tan and Gallupe, 2006) or not using
the optimal type of IT governance structure (Van Grembergen and De Haes, 2009). Our work,
however, suggests poor business results from technology may be attributed to intentional behaviors
by IT personnel to gain power or avoid its erosion. We argue that an important step in mitigating the
negative effect that power-gaining activities have on organizational performance is to recognize when
they are occurring.
By examining the contending forces affecting the different sources of power, our purpose is to
establish a better understanding of the nature of intra-organizational power for a firm’s IT department
and some of its consequences. Astley and Sachdeva (1984) observed that in the pursuit of identifying
sources of power through resource dependence, hierarchal authority is frequently overlooked. While
not the focus of this paper, the importance of formal power should not be ignored. The potential bases
of power derived from the theories utilized in this paper are by no means exhaustive or mutually
exclusive, but simply reflect broadly defined areas where intra-organizational power may be gained.
Because the theory into power developed throughout the last several decades remains relevant, we
will be revisiting this stream of research and using it as a lens for interpreting some of the challenges
faced by contemporary IT departments.
2. Literature Review
IT departments generally do not possess what Weber (1947) would refer to as “formal authority.”
However, as Mintzberg (1983) points out, in addition to the traditional view of legitimate, or hierarchal
power, power can also be gained through political or social sources. Empirical studies have examined
the perspectives of both top managers and other departments within organizations. Historically, they
suggest the perceived power of IT departments has been low (Lucas, 1984; Lucas and Palley, 1987;
Saunders and Scamell, 1986). However, more recent research in the area suggests that some IT
departments now possess moderate amounts of power (Setterstrom and Pearson, 2009), suggesting
that changes in the contemporary businesses have occurred and that this stream of research should
be revisited.
Substitutability is the degree to which there are available alternatives for completing the activities of a
particular subunit (Hickson et al., 1971). When a person or department can easily be replaced, either
through the market of available employees or from other organizations, the dispensable nature of the
subunit fails to provide a base for power. However, in instances when the functions performed by a
subunit are critical to the organization and replacement would be difficult, the subunit gains power
through its non-substitutability.
Thompson (1967) described organizations as collections of systems, both open and closed, of
interdependent roles and activities. Based on this definition, Hickson et al. (1971) describe centrality
as the degree to which a unit’s activities are connected with other units within the organization. The
concept of centrality can be divided into two distinct measures: the number of other activities within
the organization to which the workflows of the unit connect and the “speed and severity with which the
workflows of a subunit affect the final outputs of the organization.” (Hickson et al., 1971: 222). If either
the pervasiveness or immediacy of the workflows increases, the power of the subunit relative to other
units increases.
Hickson et al. (1971) hypothesized that these three contingencies (coping with uncertainty, non-
substitutability, and centrality) are interrelated to each other. Control over only a single contingency is
inadequate for gaining organizational power. Therefore, as the number and degree of contingencies
controlled by a department increases, the greater the power for the department within the
organization. Empirical testing has produced mixed results supporting this idea (Cohen and Lachman,
1988; Hinings et al., 1974; Ibarra, 1993; Saunders and Scamell, 1982).
While providing considerable explanatory power, resource dependence and strategic contingency
theories are incomplete in their assessment of the determinants of power. In his seminal discussion of
power within organizations, Pfeffer (1981) posited that affecting decision processes and consensus
among individuals were also considerable sources of organizational power. For example, while having
the authority to make a decision confers power on the decision-maker, having an effect on decision
constraints, values, information available about alternatives, and objectives used in the decision-
making process can also be interpreted as organizational power (Pfeffer, 1981). Consensus is a
situation where individual members of a subunit share a common view. Consequently, individuals
from the unit act and speak in a similar ways. This allows the subunit to argue more convincingly that
results from a decision or action are more certain, thus compelling other subunits to rely more heavily
on the focal subunit (Pfeffer, 1981). Enz (1988) offers a more compelling interpretation of how
consensus can result in power; when values and perspectives are shared between a subunit and top
management, or what Thompson (1967) would refer to as the “dominant coalition,” power will be
conferred to the subunit. Enz (1988) argues that this is a result of value systems serving as a
rationalization of conferring power to some subunits but not to other subunits.
3. Sources of power for the IT department
agreement with Brass, Butterfield and Skaggs’ (1998) work, where they argue that unethical behavior
in organizations is more likely to occur across functional areas, as opposed to occurring within a given
group. Hickson et al. (1971) discussed the possibility that individual departments will intuitively act to
preserve or gain power within an organization, thus pursuing departmental desires over organization
goals. Certainly this argument holds true for individuals and groups within the IT department as well.
Given our first two propositions, we argue that behavior previously labeled as “technology for
technology’s sake” might in fact be rationalized efforts to increase personal or IT departmental power.
Some evidence of this assertion can be seen in a longitudinal case study by Brown (1998), where
group narratives were analyzed in an effort to understand attempts to legitimize the implementation of
a hospital information support system. His research found that while the IT project team emphasized
benefits to the organization in campaigning for the implementation of the system, their underlying
motivations for system adoption were to increase the dependency on the IT department and promote
career growth. Similarly, Hussain and Cornelius (2009) conducted a longitudinal case study in which
IT management was found to implement organizational-wide IT projects to increase the credibility and
importance of the IT department.
3.2 Non-Substitutability
Thompson (1967) would describe IT as an intensive technology, as it utilizes many complex and non-
standardized inputs to provide a unique informational output. Consequently, specialists with extensive
training and experience are required to handle the various functions of the IT department. Despite the
growing prevalence of end-user development of computing solutions (Lieberman et al., 2006), a high
degree of expertise remains necessary for successfully implementing large and complex information
technologies, such as ERP systems (Maditinos, Chatzoudes, and Tsairidis, 2012).We argue that IT
department gains power through the inability of subunits to substitute its expertise with resources from
within the firm. As technology solutions become more complex and sophisticated, advanced IS skills
become increasingly important to manage them. Given that advanced IS skills are difficult to obtain or
imitate (Wade and Hulland, 2004), increasing the complexity of information technologies utilized by an
organization results in higher levels of reliance on the IT department to manage them. This view of
non-substitutability affirms Lucas and Palley’s (1987) observation that an unsophisticated level of
computing support for an organization consequently resulted in a low levels of power for the IT
department.
Proposition 3: As the complexity of IT solutions utilized by an organization increase,
the power possessed by IT department will increase.
The inundation of technology solutions in the market also has an effect on IT departmental power. As
actors within organizations gain the ability to purchase applications from external commercial sources,
they are able to rely less upon the IT department, consequently reducing the IT department’s power
(Markus and Bjorn-Andersen, 1987; Willcocks, Fitzgerald, and Lacity, 1996). We extend this
observation to assert that the availability of open source software solutions reduces reliance on a
firm’s IT department in the same way. Though organizations might choose not to seek IT solutions
externally, the availability of such solutions makes the IT department more substitutable, particularly in
cases where the technology provided by the IT department is generic and unspecialized. Conversely,
in cases where systems provided by an IT department are idiosyncratic, seeking technology solutions
through external sources becomes less viable. The more unique the features and functionalities of a
given system, the less suitable a commercially available becomes.
Proposition 4a: As IT solutions provided to an organization by an IT department
become available from sources external to the organization, the power possessed by
the IT department will decrease.
Proposition 4b: As the information systems provided to an organization become
more idiosyncratic and specialized, the power possessed by the IT department will
increase.
Interestingly, case study research has provided some evidence IT workers possess an intuitive
understanding of propositions 4a and 4b. In one particular case study, an administrator for a system
developed in-house was observed campaigning with system users to resist the implementation of a
commercially-developed ERP component (Setterstrom and Pearson, 2010). The project manager for
the ERP system believed that this action was carried out by the system administrator to conserve the
value of the unique expertise needed to manage the legacy system. Ultimately, the system
administrator’s political activity resulted in the failed implementation of the new system.
Wijk, Jansen and Lyles, 2008), we assert that some employees, IT personnel in particular, would be
motivated to actively resist the absorption of their knowledge into knowledge management systems by
the desire to retain organizational power. Research has shown that individuals’ willingness to share
knowledge can be tied to rational self-interest (Constant, Keisler, and Sproull, 1994), and our
proposition is congruent with arguments put forth by other researchers that have observed some
individuals are unwilling to contribute to knowledge management systems in an attempt to avoid a
loss of personal power (Davison, Ou, and Martinsons, 2012; Durcikova and Everard, 2004). This has
important implications, as critical activities carried by the IT function, such as information systems
development, could be negatively impacted by resistance to contribute to knowledge management
systems, because they are highly reliant on knowledge sharing for positive outcomes (Kotlarsky and
Oshri, 2005; Tesch et al., 2009).
activities is granted to divisional IT personnel and other organizational members outside the direct
hierarchal authority of the IT department (Sambamurthy and Zmud, 1999). IT governance
decentralization results in power erosion for the IT department in two ways: first, formal decision-
making authority for IT-related activities is lost by the central IT department. Second, decentralizing
the IT governance results in a reduction of dependence on the IT department for resources, such as
expertise. As dependence on the IT department decreases, so too does the power resulting from
dependence. These shifts in power provide a possible explanation for conflicts which occur when the
degree of IT governance centralization/decentralization within organizations is altered (Peterson,
2003).
Similar to IT governance decentralization, outsourcing also decreases the power of the IT department
and its members. Organizations outsource functions of their IT department to reduce costs, gain
access to technological expertise, or so that a firm might focus on its core competencies (Bergkvist
and Johansson, 2007). Despite potential benefits to the organization, attempts to outsource part of
the IT function fail remain common (Devos, Van Landeghem, and Deschoolmeester, 2008). One
possible reason for difficulties in outsourcing the IT function might be due to concerns about loss of
power. As our propositions suggest, outsourcing would erode power for several reasons, including a
reduction in the portfolio of solutions provided by the IT department and a simplification of the
information systems managed by the IT department. Members of the IT department appear to
intuitively understand these consequences, as research suggests IT personnel believe outsourcing
reduces the need for technical personnel (Apte et al., 1997). Therefore, it can be expected that IT
departments and personnel within them will be resistant to IS outsourcing (Khosrowpour et al., 2011),
including outsourcing through cloud-computing (McAfee, 2011).
3.6 Consensus
Research suggests that the psychological climate toward IT is one dimension of organizational
context which, if positive, can lead to conferring of power (Ein-Dor and Segev, 1982). When
consensus about values, goals, and organizational interests is achieved within a subunit, individuals
within the unit will behave in a similar manner. In the instance of an IT department, this would mean
that the IT professionals agree about what information technologies should be utilized and how these
technologies help the organization fulfill its goals. As a result, conflicts within the department are
minimized. This type of consensus creates a convincing image of the IT department that technology
decisions are certain, resulting in other departments perceiving the IT department as having strong
expert power.
Proposition 10: When consensus about values and goals is reached among the
members of the IT department, the power possessed by the IT department will
increase
Enz’s (1988) research suggests a correlation exists between the degree to which a department
shares critical organizational values with the firm’s dominant coalition and the power conferred to that
department. Enz cites Walsh et al. (1981) in arguing that by sharing similar values as the dominant
coalition, a department is able to influence the power structure of an organization as it is developed.
This is due to values congruity serving as a rationalizing device for top management in deciding to
distribute power to some organizational departments but not to others. Because this power
distribution is integrated into the organizational rules and structure, it becomes immobile and difficult
to erode. Furthermore, Enz posits that shared values also allow for influence in establishing which
strategic contingencies are accepted as critical to the organization. In terms of the IT department, this
can mean that IT managers are systemically granted a larger degree of power and IT is positioned as
a critical solution for coping with organizational uncertainties. Hussain and Cornelius (2009) found
evidence of this in their case study where IT management ‘demonstrated’ the benefit that a given IT
project would have to the organization, when, in fact, the ultimate goal of IT management was to
increase personal and departmental power.
Proposition 11: As the degree of value congruity between top management and the
IT department increases, the power possessed by the IT department will increase.
Frost (1987) asserted that coalitions can be an effective source of power. Historically, however, the IT
director has not been a member of the dominant coalition (Kaarst-Brown, 2005; Peppard and Ward,
1999). A powerful figure head can help increase the power of the IT department by possessing the
ability to influence the dominant coalition through strong political skills. Our assertion ties closely into
the idea of prestige power, which is the "managers' reputation in the institutional environment and
among stakeholders" (Finkelstein, 1992: 510). Prestige power for the CIO is greatest when he/she
has established relationships with key individuals in industry and the firm (Medcof, 2008).
Connections with individuals in industry help provide the CIO knowledge about technology and
access to important resources, such as highly qualified technology professionals. Such ties improve
the status of the CIO among members of the organization. At the same time, connections with the
dominant coalition allow the CIO to fully understand the organization and its strategies (Chatterjee,
Richardson, and Zmud 2001), and provide the opportunity to convey the importance of IT to the
organization in reducing uncertainty. Research provides some evidence supporting our argument, as
CIOs who reports directly to the CEO and members of the dominant coalition have greater influence
on the joint development of business and IT strategies than CIOs with no such connections (Byrd,
Lewis, and Bryan, 2006; Peppard, 2010).
Proposition 12: If the dominant coalition perceives the IT department as having a
strong leader, the power possessed by the IT department will increase.
uncertainty created by the implementation process is to a much lesser degree than that created by
more complex, sophisticated technologies. Moreover, despite potential uncertainty which is created by
implementing complex technologies frequently, if the IT implemented is consistently similar in variety,
the implementation process from project to project will also be similar. Hence, the organization gains
experience through repetition and again uncertainty is reduced as compared to the implementation
technologies that are varied in nature.
While many more potential combinations of relationships we have not discussed can have an effect
on the IT department’s overall power within the organization, our purpose is not to provide an
exhaustive set of examples. Rather, we simply aim to illustrate how possession of one source of
power, in and of itself, is insufficient for gaining intra-organizational power; an IT department must
possess a combination of power sources which results in a high degree of dependence for substantial
power to be gained.
Proposition 13: As the number and degree of valuable resources provided by the IT
department increases, the overall power possessed by the IT department will
increase.
4. Discussion
Using resource dependency theory for guidance, this paper developed a set of propositions
describing factors that influence the power relationship between an IT department and other actors
within an organization. Further, we suggest that these sources of power are interrelated; as the
number and the degree to which these sources are controlled by a department increases, the greater
the power possessed. With the increasing prominence of IT in organizations, initial intuition would
suggest that the IT department’s power would correlate with the general increase of spending on IT.
However, this assumption, in and of itself, is short-sighted. While the growth of IT usage potentially
acts to increase the IT department’s power within an organization, there are also many counter acting
forces, such as increased user self-efficacy, the increasing availability of commercial technology, and
the information absorption by organizations.
Organizations experience many problems related to their IT functions. These issues commonly
include technology for technology’s sake, a lack of user preferences being integrated into IT systems
development projects, and lack of IT alignment with overall firm strategy. With the understanding that
organizations are places that political activity takes place and that individual employees’ preferences
and needs are not necessarily consistent the needs of the organization, we argue that many of the
common IT issues organization grapple with might be, in part, the result of intentional behavior by the
IT personnel to avoid power erosion.
Table 1: IT Departmental Power Propositions
Coping with Uncertainty
Proposition 1: As an organization’s utilization of information technology to store, retrieve,
create, and transmit information increases, the power possessed by the IT department will
increase.
Proposition 2: As the frequency in which new information technologies are implemented in
an organization increases, the power possessed by the IT department will increase.
Non-Substitutability
Proposition 3: As the complexity of IT solutions utilized by an organization increase, the
power possessed by IT department will increase.
Proposition 4a: As IT solutions provided to an organization by an IT department become
commercially available, the power possessed by the IT department will decrease.
Proposition 4b: As the information systems provided to an organization become more
idiosyncratic and specialized, the power possessed by the IT department will increase.
Proposition 5: As computer users in an organization become more self-efficacious with IT,
the power possessed by the IT department decreases.
Proposition 6: As tasks executed by the IT department become routinized and documented,
the power possessed by the IT department will decrease.
Centrality in Workflow
Proposition 7: As the workflow of organizational processes becomes more dependent on
information technology to administer, the power possessed by the IT department will
increase.
Provide Resources
Proposition 8a: As the variety of IT systems utilized by an organization increases, the power
possessed by the IT department will increase.
Proposition 8b: As the number of IT systems utilized by an organization increases, the
power possessed by the IT department will increase.
Affecting Decision Processes
Proposition 9: As the degree to which IT managers affect the decision-making processes
regarding organizational IT increases, the power possessed by the IT department increases.
Consensus
Proposition 10: When consensus about values and goals is reached among the members of
the IT department, the power possessed by the IT department will increase
Proposition 11: As the degree of value congruity believed by top management to exist
between itself and the IT department increases, the power possessed by the IT department
will increase.
Proposition 12: If the dominant coalition perceives the IT department as having a strong
figure head, the power possessed by the IT department will increase.
Practitioners have many possible solutions to help mitigate these dysfunctional behaviors, but an
While our discussion of implications for our propositions focused on how the pursuit of power by IT
personnel can be detrimental to organizational outcomes, caution should be exercised interpreting the
arguments we have presented. As previously discussed, power and politics have long been observed
as having an important role in the execution of business functions by the IT department, such as the
implementation of new information systems into an organization (Avgerour and McGrath, 2007;
Doolin, 2004; Jasperson et al., 2002; Lapointe and Rivard, 2005; Levine and Rossmoore, 1995;
Markus and Bjorn-Andersen, 1987; Sillince and Mouakket, 1997; Silva and Hirschheim, 2007).
Consequently, while power may be a source of dysfunction for an organization’s IT function, it is
nonetheless crucial in garnering commitment and support for IT strategy and decisions, as well as
facilitating change.
Empirical research into this area may be problematic in that, as discussed earlier, gauging power can
be difficult. To address this issue, researchers have suggested that both objective and subjective
measures be used in combination when measuring power (March, 1966; Pfeffer, 1981). Provan
(1980) provides one methodology which might be helpful in overcoming these difficulties by
suggesting measures of power that can be triangulated to an overall power evaluation; specifically, he
suggests gathering perceptual data, information about formal position within an organization and
membership in decision-making coalitions, information on dependence, and information on
demonstrated ability to influence organizational outcomes, such as budget allocations. If information
gathered from these multiple measures of power is consistent, it can be argued that an accurate
overall measure of power for a department has been obtained. Even if a clear assessment IT
departmental power can be obtained, the meaning of such data may be difficult to determine due to
the substantial differences in the types of IT utilized from organization to organization. Therefore, we
suggest either a multi-case study or a longitudinal study of a single organization. Using this approach,
researchers can compare the overall assessment of IT departmental power across cases or time,
then identify variations in potential sources of power that might be the cause of these power
differences. This method of theory testing is referred to as “pattern matching” (Campbell, 1975). In our
opinion, a longitudinal study of a single organization would provide the highest degree of internal
validity, as this approach would best control confounding variables.
4.3 Conclusions
Power will remain an important consideration in the study of both organization theory and
management information systems, due to its pervasive nature in the social systems that make up
organizations. A more complete understanding of power will help the explanation of behaviors, both at
the individual and departmental level, which otherwise appear to have no rational cause.
References
Agarwal, R., Sambamurthy, V. and Stair, R.M. (2000) ‘Research report: The evolving relationship between
general and specific computer self-efficacy – An empirical assessment’, Information Systems Research, vol.
11, pp. 418-430.
Alvarez, R. (2008) ‘Examining technology, structure and identity during an enterprise system implementation’,
Information Systems Journal, vol. 18, pp. 203-224.
Apte, U.M., Sobel, M.G., Hanaoka, S., Shimada, T., Saarinen, T., Salmela, T. and Vepsalainen, A.P.J. (1997) ‘IS
outsourcing practices in the USA, Japan, and Finland’, Journal of Information Technology, vol. 12, pp. 289-
304.
Astley, W.G. and Sachdeva, P.S. (1984) ‘Structural sources of intraorganizational power: A theoretical synthesis’,
Academy of Management Review, vol. 9, pp. 104-113.
Avgerour, C and McGrath, K. (2007) ‘Power, rationality, and the art of living through socio-technical change’, MIS
Quarterly, vol. 31, pp. 295-315
Barker, T. and Frolick, M.N. (2003) ‘ERP implementation failure: A case study’, Information Systems
Management, vol. 20, pp. 43-49.
Bergkvist, L. and Johansson, B. (2007) ‘Evaluating motivational factors involved at different stages in an IS
outsourcing decision process’, Electronic Journal of Information Systems Evaluation, vol. 10, pp. 23-30.
Brass, D.J., Butterfield, K.D. and Skaggs, B.C. (1998) ‘Relationships and unethical behavior: A social network
perspective’, Academy of Management Review, vol. 23, pp. 14-31.
Brown, A.D. (1998) ‘Narrative, politics and legitimacy in an IT implementation’, Journal of Management Studies,
vol. 35, pp. 35-58
Buchman, D.A. (2008) ‘You stab my back, I’ll stab yours: Management experience and perceptions of
organization political behavior’, British Journal of Management, vol. 19, pp. 49-64.
Byrd, T.A., Lewis, B.R. and Bryan, R.W. (2006) ‘The leveraging influence of strategic alignment on IT investment:
An Empirical examination’, Information & Management, vol. 43, pp. 308-321.
Campbell, D.T. (1975) ‘Degrees of freedom and the case study’, Comparative Political Studies, vol. 8, pp. 178-
193.
Carton, F. and Adam, F. (2010) ‘Towards a model for determining the scope of ICT integration in the enterprise:
The case of enterprise resource planning (ERP) systems’, Electronic Journal of Information Systems
Evaluation, vol. 13, pp. 17-26.
Chatterjee, D., Richardson, V.J. and Zmud, R.W. (2001) ‘Examining the shareholder wealth effects of
announcements of newly created CIO positions’, MIS Quarterly, vol. 25, pp. 43-70
Cohen, I. and Lachman, R. (1988) ‘The generality of the strategic contingencies approach to sub-unit power’,
Organization Studies, vol. 9, pp. 371-391.
Constant, D., Keisler, S. and Sproull, L. (1994) ‘What’s mine is ours, or is it? A study of attitudes about
information sharing’, Information Systems Research, vol. 5, pp. 400-421.
Crozier, M. (1964) The Bureaucratic Phenomenon, Chicago, IL: University of Chicago Press.
Cyert, R. and March, J. (1963) A Behavioral Theory of the Firm, Englewood Cliffs, NJ: Prentice-Hall.
Dalcher, D. and Genus, A. (2003) ‘Introduction: Avoiding IS/IT implementation failure’, Technology Analysis and
Strategic Management, vol. 15, pp. 403-407.
Davison, R.M., Ou, C.X.J. and Martinsons, M.G. (2012) ‘Information technology to support informal knowledge
sharing’, Information Systems Journal, doi: 10.1111/j.1365-2575.2012.00400.x.
Devos, J., Van Landeghem, H. and Deschoolmeester, D. (2008) ‘Outsourced information systems failures in
SMEs: A multiple case study’, Electronic Journal of Information Systems Evaluation, vol. 11, pp. 73-82.
Doolin, B. (2004) ‘Power and resistance in the implementation of a medical management information system’,
Information Systems Journal, vol. 14, pp. 343-362.
Downey, J.P. and Zeltmann, S. (2009). ‘The role of competence level in the self-efficacy skills relationship: An
empirical examination of the skill acquisition process and its implications for information technology training’,
International Journal of Training and Development, vol. 13, pp. 96-110.
Durcikova, A. and Everard, A. (2004) ‘Managing the corporate zoo, A knowledge management perspective’,
Electronic Journal of Knowledge Management, vol. 2, pp. 13-18.
Ein-Dor, P. and Segev, E. (1982) ‘Organizational context and MIS structure: Some empirical evidence’, MIS
Quarterly, vol. 6, pp. 55-68.
Emerson, R.M. (1962) ‘Power-dependence relations,’ American Sociological Review, vol. 27, pp. 31-41.
Enz, C.A. (1988) ‘The role of value congruity in intraorganizational power’, Administrative Science Quarterly, vol.
33, pp. 284-304.
Finkelstein, S. (1992) ‘Power in top management teams: Dimensions, measurement, and validation’, Academy of
Management Journal, vol. 35, pp. 505-538
French, J.R.P. and Raven, B. (1959) ‘Bases of social power’, in Cartwright, D. (ed.) Studies in Social Power, Ann
Arbor, MI: University of Michigan Press.
Frost, P.J. (1987) ‘Politics, power, and influence’, in Jablin, F.M., Putnam, L.L., Roberts, K.H. and Porter, L.W.
(ed.) The Handbook of Organizational Communication: An Interdisciplinary Perspective, Newbury Park, CA:
Sage Publications.
Fulk, J. and DeSanctis, G. (1995) ‘Electronic communication and changing organizational forms’, Organizational
Science, vol. 6, pp. 337-349.
Gamson, W.A. (1968) Power and discontent, Homewood, IL: Dorsey Press.
Halawi, L.A., Aronson, J.E. and McCarthy, R.V. (2005) ‘Resource-based view of knowledge management for
competitive advantage’, Electronic Journal of Knowledge Management, vol. 3, pp. 75-86.
Hawari, A. and Heeks, R. (2010) ‘Explaining ERP failure in a developing country: A Jordanian case study’,
Journal of Enterprise Information Management, vol. 23, pp. 135-160.
Hickson, D.J., Hinings, C.R., Lee, C.A., Schneck, R.E. and Pennings, J.M. (1971) ‘A strategic contingencies’
theory of intraorganizational power’, Administrative Science Quarterly, vol. 16, pp. 216-229.
Higgs, M. (2009) ‘The good, the bad and the ugly: Leadership and narcissism’, Journal of Change Management,
vol. 9, pp. 165-178.
Hinings, C., Hickson, D.J., Pennings, J.M. and Schneck, R.E. (1974) ‘Structural conditions of intraorganizational
power’, Administrative Science Quarterly, vol. 19, pp. 22-44.
Hitt, L.M., Wu, D.J. and Zhou, X. (2002) ‘ERP investment: Business impact and productivity measures’, Journal
of Management Information Systems, vol. 19, pp. 71-98.
Howcroft, D. and Light, B. (2006) ‘Reflections on issues of power in packaged software selection’, Information
Systems Journal, vol. 16, pp. 215-235.
Hussain, Z.I. and Cornelius, N. (2009) ‘The use of domination and legitimation in information systems
implementation’, Information Systems Journal, vol. 19, pp. 197-224.
Ibarra, H. and Andrews, S.B. (1993) ‘Power, social influence, and making sense: Effects of network centrality and
proximity on employee perceptions’, Administrative Science Quarterly, vol. 38, pp. 277-303.
Ibarra, H. (1993). Network Centrality, Power, and Innovation Involvement: Determinants of Technical and
Administrative Roles. Academy of Management Journal, vol. 36, pp. 471-501.
in ‘t Hout, R, Vrancken, J. and Schrijnen, P. (2010) ‘Wiki-based knowledge management in transportation
consultancy, a case study’, Electronic Journal of Information Technology Evaluation, vol. 13, pp. 133-142.
Jasperson, J., Carte, T.A., Saunders, C.S., Butler, B.S., Croes, H.J.P. and Zheng, W. (2002) ‘Review: Power and
information technology research: A metatriangulation review’, MIS Quarterly, vol. 26, pp. 397-459.
Kaarst-Brown, M. L. (2005) ‘Understanding an organization’s view of the CIO: The role of assumptions about IT’,
MIS Quarterly Executive, vol. 4, pp. 287–301.
Kankanhalli, A., Tan, B.C.Y. and Wei, K.K. (2005) ‘Contributing knowledge to electronic repositories: An empirical
investigation’, MIS Quarterly, vol. 29, pp. 113-143.
Khosrowpour, M., Subramaniam, G.H., Gunderman, J. and Saber, A. (2011) ‘Managing information technology
with outsourcing: An assessment of employee perceptions’, Journal of Applied Business Research, vol. 12,
pp. 85-96.
King, J.L. (1983) ‘Centralized versus decentralized computing: Organizational considerations and management
options’, Computing Surveys, vol. 15, pp. 319-349.
Kohli, R. and Devaraj S. (2003) ‘Measuring information technology payoff: A meta-analysis of structural variables
in firm-level empirical research’, Information Systems Research, vol. 14, pp. 127-145.
Kotlarsky, J. and Oshri, J. (2005) ‘Social ties, knowledge sharing, and successful collaboration in globally
distributed development projects’, European Journal of Information Systems, vol. 14, pp. 37-48.
Lapointe, L. and Rivard, S. (2005) ‘A multilevel model of resistance to information technology implementation’,
MIS Quarterly, vol. 29, pp. 461-491.
Levine, H.G. and Rossmoore, D. (1994-95) ‘Politics and the function of power in a case study of IT
implementation’, Journal of Management Information Systems, vol. 11, pp. 115-133.
Lieberman, H., Paterno, F., Klann, M. and Wulf, V. (2006) ‘End-user development: An emerging paradigm’, in
End-User Development, Dordrecht: Springer, pp. 1-8.
Lucas, H.C. and Palley, M.A. (1987) ‘Plant management, the locus of decision making, and the information
services department’, Journal of Management Information Systems, vol. 3, pp. 34-48.
Lucas, H.C. (1984) ‘Organizational power and the information services department’, Communications of the
ACM, vol. 27, pp. 58-65.
Maditinos, D., Chatzoudes, D. and Tsairidis, C. (2012) ‘Factors affecting ERP system implementation
effectiveness’, Journal of Enterprise Information Management, vol. 25, pp. 60-78.
March, J. G. (1966) ‘The power of power’, in Easton, D. (ed.) Varieties of political theory, Englewood Cliffs, NJ:
Prentice-Hall.
March, S.T. and Hevner, A.R. (2007) ‘Integrated decision support systems: A data warehousing perspective’,
Decision Support Systems, vol. 43, pp. 1031-1043.
Markus, M.L. and Bjorn-Andersen, N. (1987) ‘Power over users: Its exercise by system professionals’,
Communications of the ACM, vol. 30, pp. 498-504.
McAfee, A. (2006) ‘Mastering the three worlds of information technology’, Harvard Business Review, vol. 84, pp.
141-149.
McAfee, A. (2011) ‘What every CEO needs to know about the cloud’, Harvard Business Review, vol. 89, pp. 124-
132.
McCall, M.W. Jr. (1979) ‘Power, authority, and influence’, in Kerr, S. (ed.) Organizational behavior, Columbus,
OH: Grid Publishing.
McKeen, J.D. and Smith, H.A. (1997) ‘Management challenges in IS: Successful strategies and appropriate
action. New York, NY: John Wiley & Sons.
Medcof, J.W. (2008) ‘The organizational influence of the chief technology officer’, R&D Management, vol. 38, pp.
406-420.
Mintzberg, H. (1983) Power in and around organizations, Englewood Cliffs, NJ: Prentice-Hall.
Muscatello, J.R. and Parente, D.H. (2006) “Enterprise resource planning (ERP): A post implementation cross-
case analysis”, Information Resources Management Journal, vol. 19, pp. 61-80
O’Donovan, B., Seymour, L., Geldenhuys, J., Isaacs, M. and Kaulule, K. (2010) ‘The influence of organizational
memory mismatched and coping strategies on ERP outcomes’, Electronic Journal of Information Systems
Evaluation, vol. 13, pp. 165-176.
Orlikowski, W.J. and Gash, D.C. (1994) ‘Technological frames: Making sense of information technology in
organizations’, ACM Transactions of Information Systems, vol. 12, pp. 174-207.
Peppard, J. (2010) ‘Unlocking the performance of the chief information officer (CIO)’, California Management
Review, vol. 52, pp. 73-99.
Peppard, J. and Ward, J. (1999) ‘‘Mind the gap’: Diagnosing the relationship between the IT organisation and the
rest of the business’, Journal of Strategic Information Systems, vol. 8, pp. 29-60.
Peterson, R.R. (2003) ‘Integrating strategies and tactics for information technology governance’, in Grembergen,
W.V. (ed.) Strategies for information technology governance, Hershey, PA: Idea Group Publishing.
Pettigrew, A.M. (1972) ‘Information control as a source of power’, Sociology, vol. 6, pp. 187-204.
Pfeffer, J. and Salancik, G.R. (1974) ‘Organizational decision making as a political process: The case of a
university budget’, Administrative Science Quarterly, vol. 19, pp. 135-151.
Pfeffer, J. and Salancik, G.R. (1978) The external control of organizations: A resource dependence perspective,
New York, NY: Harper & Row.
Pfeffer, J. (1981) Power in organizations, Marshfield, MA: Pitman Publishing.
Pfeffer, J. (1989) ‘Political perspective on careers: Interests, networks, and environments’, in Arthur, M.B. and
Lawrence, B.S. (ed.) Handbook of career theory, New York, NY: Cambridge University Press.
Preston, D.S., Chen, D. and Leidner, D.E. (2008) ‘Examining the antecedents and consequences of CIO strategic
decision-making authority: An empirical study’, Decision Sciences, vol. 39, pp. 605-642.
Provan, K.G. (1980) ‘Recognizing, measuring, and interpreting the potential/enacted power distinction in
organizational research’, Academy of Management Review, vol. 5, pp. 549-559.
Reed, M.I. (1996) ‘Expert power and control in late modernity: An empirical review and theoretical synthesis’,
Organization Studies, vol. 17, pp. 573-597.
Salancik, G.R. and Pfeffer, J. (1974) ‘The basis and use of power in organizational decision making: The case of
a university’, Administrative Science Quarterly, vol. 19, pp. 453-473.
Sarker, S., Sarker, S. and Sidorova, A. (2006) ‘Understanding business process change failure: An actor-network
perspective’, Journal of Management Information Systems, vol. 23, pp. 51-86.
Sambamurthy, V. and Zmud, R.W. (1999) ‘Arrangements for information technology governance: A theory of
multiple contingencies’, MIS Quarterly, vol. 23, pp. 261-290.
Saunders, C.S. and Scamell, R.W. (1982) ‘Intraorganizational distributions of power: Replication research’,
Academy of Management Journal, vol. 25, pp. 192-200.
Saunders, C.S. and Scamell, R.W. (1986) ‘Organizational power and the information services department: A re-
examination’, Communications of the ACM, vol. 29, pp. 142-147.
Setterstrom, A.J. and Pearson, J.M. (2009) ‘Resource dependence: A case study of power and the information
technology department’, 2009 International Decision Sciences Institute Meeting (DSI), New Orleans,
Louisiana.
Setterstrom, A.J. and Pearson, J.M. (2010) ‘Grasping for power: A multi-case study of dysfunctional behaviors by
information technology personnel’, 2010 International Decision Sciences Institute Meeting (DSI), San Diego,
California.
Sillince, J.A.A. and Mouakket, S. (1997) ‘Varieties of political process during systems development’, Information
Systems Research, vol. 8, pp. 368-397.
Silva, A. (2007) ‘Epistemological and theoretical challenges for studying power and politics in information
systems’, Information Systems Journal, vol. 17, pp. 165-183.
Silva, L. and Hirschheim, R. (2007) ‘Fighting against windmills: Strategic information systems and organizational
deep structures’, MIS Quarterly, vol. 31, pp. 327-354.
Stohr, E.A. and Zhao, J.L. (2001) ‘Workflow automation: Overview and research issues’, Information Systems
Frontiers, vol. 3, pp. 281-296.
Tan, F.B. and Gallupe, R.B. (2006) ‘Aligning business and information systems thinking: A cognitive approach’,
IEEE Transactions on Engineering Management, vol. 53, pp. 223-237
Tesch, D., Sobol, M.G., Klein, G. and Jiang, J.J. (2009) ‘User and developer common knowledge: Effect on the
success of information system development projects’, International Journal of Project Management, vol. 27,
pp. 657-664.
Thompson, J.D. (1967) Organizations in action, New York: McGraw-Hill.
Torkzadeh, G., Chang, J.C.J. and Demirhan, D. (2006) ‘A contingency model of computer and Internet self-
efficacy’, Information & Management, vol. 43, pp. 541-550.
Van Grembergen, W. and De Haes, S. (2009) Enterprise governance of information technology: Achieving
strategic alignment and value, New York, NY: Springer.
van Wijk, R., Jansen, J.J.P. and Lyles, M.A. (2008) ‘Inter- and intra-organizational knowledge transfer: A meta-
analytic assessment of its antecedents and consequences’, Journal of Management Studies, Vol. 45, pp.
830-853.
Wade, M. and Hulland, J. (2004) ‘Review: The resource-based view and information systems research: Review,
extension, and suggestions for future research’, MIS Quarterly, vol. 28, pp.
Walsh, K., Hinings, B., Greenwood, R. and Ranson, S. (1981) ‘Power and advantage in organizations’,
Organization Studies, vol. 2, pp. 131-452.
Weber, M. (1947) The theory of social and economic organization, New York, NY: Free Press.
Weill, P. and Woodham, R. (2002) ‘Don’t just lead, govern: Implementing effective IT governance’, Sloan School
of Management, CISR Working Paper No. 326.
Weimann, P., Hinz, C., Scott, E. and Pollock, M. (2010) ‘Changing the communication culture of distributed
teams in a world where communication is neither perfect nor complete’, Electronic Journal of Information
Systems Evaluation, vol. 13, pp. 187-196.
Willcocks, L., Fitzgerald, G. and Lacity, M. (1996) ‘To outsource IT or not?: Recent research on economics and
evaluation practice’, European Journal of Information Systems, vol. 5, pp. 143-
Wrong, D.H. (1968) ‘Some problems in defining social power’, The American Journal of Sociology, vol. 73, pp.
673-681.
Xue, Y., Liang, H. and Boulton, W.R. (2008) ‘Information technology governance in information technology
investment decision processes: The impact of investment characteristics, external environment, and internal
control’, MIS Quarterly, vol. 32, pp. 67-96.
Yukl, G. (2002). Leadership in Organizations, Upper Saddle River, NJ: Prentice Hall.
Zammuto, R.F., Griffith, T.L., Majchrzak, A., Dougherty, D.J. and Faraj, S. (2007) ‘Information technology and the
changing fabric of organization’, Organization Science, vol. 18, pp. 749-762.
Zhao, J.L., Kumar, A. and Stohr, E.A. (2000-2001) ‘Workflow-centric information distribution through E-mail’,
Journal of Management Information Systems, vol. 17, pp. 45-72.
Abstract: In this paper we try to assess the impact of ERP-implementations on the development of
non-financial organizational performance, as described by Shang and Seddon (2002) and Eckartz et
al. (2009). We assess this impact for Dutch small and medium-sized enterprises, using a small but
unique dataset. Several aspects of the performance of organizations are compared before and after
the introduction of an ERP-system, taking into account a three-year period, and controlling for several
influential factors (like organizational size, financial health and sectoral differences). We conclude that
by and large, organizational performance increased significantly more for organizations that
implemented an ERP-system in the last three years than for organizations that did not implement
such a system. We also conclude that organizations that implemented an ERP-system at most three
years ago did not have significantly lower non-financial performance than organizations that did not
implement such a system. Additional analyses suggest that we would oversell our results if we would
claim that ERP-systems are the main or sole source of the effects found. Nevertheless, although
limited to Dutch SMEs, our results contradict some of the views expressed in the ERP-literature.
1. Introduction
Ever since their introduction in the 1990s, Enterprise Resource Planning (ERP) systems have been
widely used by organizations wishing to work with integrated information systems in the hopes to
increase their market agility (Grabski and Leech 2007; Keller 1999). Many researchers have tried to
assess the impact of the introduction of ERP- systems on organizational conduct, often focusing on a
system’s performance effects. Examples include Hunton et al. (2003), Kallunki et al. (2011), Liu et
al.(2008), Poston and Grabski (2001), Nicolaou (2004) and Nicolaou and Bhattacharya (2006). One of
the main questions in this type of research seems to be whether the (chiefly) financial performance of
organizations adopting an ERP-system has deteriorated or improved during the post-implementation
period.
In general, organizations (and consultancy firms alike) seem to think that the introduction of an ERP-
system allows for more efficiency in organizational work, and therefore, for better (financial)
performance vis-à-vis non-adopting organizations, because of the fact that best practices are
embraced (Bernroider 2008; Davenport 1998; Sneller 2010). It is also expected that in conjunction to
this, ERP-implementations invoke more reliable information recording and exchanges in an
organization (Shang and Seddon 2002). However, is the situation always that simple?
Markus and Tanis (2000) developed a framework to describe the ‘typical’ phases involved in the
adoption and implementation of an ERP-system. After a chartering phase, comprising the decisions
leading to the approval of the implementation of a specific system, the communication in the
organization about this, and the system’s funding, a project phase is entered, in which a system is set
up and executed in one or more organizational (business) units. This ultimately results in its rollout
and start-up across the organization (‘going live’). Thereafter, a shakedown phase is witnessed, which
takes up the period of time between the aforementioned ‘go live’ date and the moment that user
acceptance has occurred, bugs have been fixed, system training has been accommodated, and a
system has been ‘fine-tuned’ to fit an organization. Finally, Markus and Tanis discern an onward and
upward phase. This phase generally occurs between 1-3 years after a system’s ‘go live’ date, and
covers the period from ‘normal operations’ until a system is replaced completely by another system or
is upgraded. Alas, all of these phases are fraught with problems that can affect an organization’s
performance. For instance, Kumar et al. (2002) found that in many Canadian governmental
organizations, project schedules tended to be revised as organizations underestimated the amount of
work involved in implementing an ERP-system, inadequate training in the new system occurred, and
difficulties in assuring the quality of the data entered in the system were paramount. Other studies
have reported similar problems: implementation costs may rise exponentially, employees may refuse
to work with the new system, and data integration processes between new and old (legacy) systems
may be more troublesome than expected (Botta-Genoulaz and Millet 2005; Hunton et al. 2003; Ross
and Vitale 2000; Nicolaou 2004; Scott 1999; Sneller 2010). It may therefore not come as a surprise
that more than 70% of ERP-introductions do not reach their intended effects, and may even be
regarded as complete failures (Al-Mashari et al. 2006; Buckhout et al. 1999; Hong and Kim 2002;
Stefanou 2001).
In this paper we will look at organizations that implemented an ERP-system between one and three
years ago, and compare their self-assessed organizational benefits with a set of comparable
organizations that did not implement or have such systems in the same period. The period of three
years has been chosen for Nicolaou and Bhattacharya (ibid.) concluded that organizations adopting
an ERP-system needed at least two years to generate positive financial performance. The authors
term this the “... lag-led re-emergence of performance gains ...” (ibid. p20) (see also Wah 2000).
Contrary to their research, however, we decide to look at non-financial organizational performance
(Shang and Seddon 2002), this being an area where so far little research seems to have been done
(Eckartz et al. 2009). On top of this, we follow up on Esteves and Bohorquez’s (2007) call for more
research on the benefits of ERP-implementations in small and medium-sized enterprises, this being
the market where ERP-systems are nowadays implemented most often (Adam and O’Doherty 2003).
This paper will proceed as follows. We start by setting out the types of benefits that may be expected
from an ERP-system in section 2.. Thereafter, in section 3.1, our research approach and method will
be discussed. The main results will be presented in section 3.2. Finally, in section 4, we will put
forward the conclusions that we think can be distilled from this research and discuss some of its
drawbacks. We will also point out what we see as viable avenues for further research.
2. ERP benefits
In this study we define an ERP-system as a business support system that maintains the data needed
for various business functions such as manufacturing, financials and supply chain management in a
single database, so that transactions only need to be processed once (Kumar and van Hillegersberg
2000).
Eckartz et al. (2009) present the results of an extensive literature review on enterprise system (ES)
performance, with particular reference to ERP-systems. They aim to determine all benefits that may
be achieved during and after an ES-implementation, both tangible and intangible. Their analysis
results in what they call a ‘3-dimensional benefit framework’. It is largely based on a similar
classification of benefits from Shang and Seddon (2002). The three dimensions Eckartz et al. (2009)
distinguish are:
Operational, managerial and strategic benefits (Anthony 1965; Shang and Seddon 2002);
Process, customer, financial and innovation benefits, following the four perspectives of the
balanced scorecard (Kaplan and Norton 1993). Also included is a fifth perspective to assess
employee resistance or willingness to change (Eckartz et al. 2009);
Benefits falling into IT infrastructure and organizational categories (Shang and Seddon 2002).
These are often intangible and hard to identify (such as organizational learning and improvement
in communications), and are, consequently, not researched very much (Eckartz et al. 2009).
Eckartz et al. explicitly ask researchers to validate their framework in a variety of ways, among others
by determining how the various dimensions and categories of ES-related benefits impact on one
another. Our own goal with their framework is, however, more modest. We wish to examine (non-
financial) organizational performance, which Eckartz et al. call ‘organizational benefits’. We will only
try to validate this particular aspect of their framework, as these benefits apparently have not been
researched as often as more tangible ES-benefits (see also Hunton et al. 2003; Nicolaou and
Bhattacharya 2006). According to Shang and Seddon (2002), organizational benefits “… arise when
the use of an ES benefits an organization in terms of focus, cohesion, learning and execution of its
chosen strategies” (p279). They go on to argue that the potential organizational benefits of ES-
implementations evolve around the following issues, which will all be taken along in this study:
Changing work patterns with shifted focus: the harmonization of interdepartmental processes and
interdisciplinary matters
Facilitating business learning and broadening of employee skills: greater possibilities to enhance
employee learning
Employee empowerment: more pro-active, perhaps even entrepreneurial, employee behavior and
involvement in business management
Building common visions: departments work as a unit, and not as separate entities, sustaining a
shared image on organizational work across different levels of the organization
Shifting work focus: concentration on core work
Increased employee morale and satisfaction: increased work efficiency and (more) content users,
who are provided with better quality service.
Nicolaou and Bhattacharya (2006) report that changes in ERP-systems often offset implementation
issues that at first, negatively affect the long-term financial performance of organizations. The sooner
adaptations are made, the better performance may develop. Adapting a system too late may result in
a deterioration of financial performance. Following Nicolaou (2004), they also find that an, at least,
two year time lag may be useful to assess the performance impact of ERP-systems, as this lag seems
to be necessary for positive differential financial performance in adopting vis-à-vis non-adopting
organizations to start materializing. Kallunki et al. (2011) obtain similar results in their exploration of
the effects of ERP-implementations on both the financial and non-financial performance of 70 Finnish
business-units. They conclude that ERP-systems can improve both the financial and non-financial
performance of business-units, chiefly in the long run (three years or more). Using specific forms of
management control can help to achieve this.
Taken together, this leads to the first hypothesis we would like to test in this study:
H1: Organizational benefits are significantly larger for organizations that have
implemented an ERP-system in the last three years than for organizations that did not
implement or have such a system.
The financial performance effects of ERP-implementations in Chinese chemical firms were assessed
by Liu et al. (2008). They find no significant performance improvement during the implementation
period and during the first three years after implementation. At first, a decline in performance is
witnessed, which is in line with the Markus and Tanis (2000) framework and previous studies by
Nicoloau and Bhattacharya (2006) among others. However, a slight performance improvement in the
third year after implementation occurs, which may indicate that the financial benefits of ERP-
implementations may only manifest themselves after more extensive ERP-use.
The impact of ERP-system adoption on the financial performance of over 60 organizations that were
matched, on the basis of their 2-digit SIC-code as well as their size, with organizations that had not
adopted such a system, was the focus of a study conducted by Hunton et al. (2003). Although the
financial performance of ERP-adopting organizations was generally better over a three year period
than for non-adopters, no improvement in the financial performance of adopting organizations could
be witnessed before and after the implementation of an ERP-system. Nevertheless, the financial
performance of non-adopters decreased over time.
These particular outcomes lead up to the following hypothesis that will be tested in this study:
H2: Organizations that implemented an ERP-system in the last three years had significantly better
organizational benefits three years ago than organizations that did not implement or have such a
system (Hunton et al. 2003).
Esteves and Bohorquez (2007) stress the importance of more research on the benefits of ERP-
implementations in small and medium-sized enterprises. This call is followed up on here. We will
compare differences in organizational benefits for Dutch small and medium-sized enterprises.
Bernroider and Tang (2003), albeit indirectly, suggest that although many studies have been
conducted on ERP-implementation effects in Europe, few studies have been conducted in the
Netherlands. As ERP-systems as such have been said to have their roots in Europe (Pairat and
Jungthirapanich 2005), the lack of focus on this country is striking. It will be ‘redeemed’ here.
We will now discuss our research approach. We will start with setting out our research method, after
which a description of the main research findings will follow.
3. Research approach
3.1 Method
The data used to test H1 and H2 have been obtained via a self-developed survey that was almost
entirely based on validated concepts taken from the ERP-related literature (Berchet and Habchi 2005;
Bradley 2008; Eckartz et al. 2009, Grabski and Leech 2007; Hong and Kim 2002; Hunton et al. 2003;
Shang and Seddon 2002; Soja and Poliwoda-P kosz 2009; Vluggen 2006). The survey was
developed to assess the performance and circumstances faced by a variety of Dutch firms during the
2007-2009 period. Two surveys were constructed: one for organizations that had implemented an
ERP-system in the last three years (and had no such system before), and a similar survey for
organizations that did not have an ERP-system - the main difference being that the latter survey came
without the questions on ERP.
As part of the preparation for their master of science thesis, part-time master of science students in
accountancy from the university that the authors of this paper stem from were asked to select and
approach top-level managers who were knowledgeable about an organization’s primary processes
(CFOs, CIOs, etc.) in four organizations: two organizations with and two organizations without an
ERP-system. They received the, for them, appropriate survey. The data were collected in November
and December 2010, and were returned both on paper and in a pre-prepared Excel-sheet. As names
and contact persons of the organizations that had been approached had to be handed in as well, we
had the possibility to check whether the organizations in question had indeed been approached, and
an appropriate interviewee had been selected. Several checks were carried out to safeguard that the
data had been entered correctly in Excel (for instance, by checking several surveys in their entirety or
1
by checking specific variables across surveys) . The survey questions relevant to this study can be
found in Appendix 1. Forty organizations in our sample were SMEs (according to EC guideline
2003/361/EC 2) and completed the survey they had been given. The data of these organizations will
be used in our analyses. The aim is to create matched pairs of organizations based on their size,
whose organizational benefits can subsequently be compared as stated in H1 and H2. We will correct
the matching procedure for sectoral effects and possible other factors related to an organization’s
size. Like in the Hunton et al. (2003) study, sectoral effects will be taken into account by looking at an
1
Both surveys can be obtained from the corresponding author on request.
2
Refer to http://ec.europa.eu/enterprise/policies/sme/facts-figures-analysis/sme-definition/index_en.htm for details.
organization’s 2-digit SIC-code (see also Barber and Lyon 1996). Size is expressed in this study as
the natural logarithm of the sales of an organization, which is one of the ways in which Hunton et al.
(2003) operationalized this item.
However, contrary to the Barber and Lyon (1996) and Hunton et al. (2003) studies, we believe that
organizational size is dependent on a number of other factors, the influence of which has to be purged
before matching can occur. To accomplish this, two regressions will be run. We first run the following
regression for organizations that had implemented an ERP-system in the last three years:
where 3:
LOGMSALES = the natural logarithm of the mean of organizational sales for the 2007-2009 period
(our size measure);
FINHEALTH = financial health of an organization;
PEU = perceived environmental uncertainty;
GOLIVE = dummy indicating whether the ‘go live’ date of a system occurred in 2007 (‘0’), 2008 (‘1’),
or 2009 (‘2’).
The reason for including the ‘go live’ date of a system has been set out in the previous section: it
generally takes time before the fruits of an ERP-implementation can be reaped (Markus and Tanis
2000; Nicolaou and Bhattacharya 2006).The PEU variable has been incorporated to accommodate
effects occurring in the environment of an organization. The operationalization of this variable has
been taken from Vluggen (2006). Cronbach’s alpha will be calculated to substantiate the
operationalization chosen.
The financial health variable tries to put the respondents’ view on whether they thought their
organization was in a healthy financial state. This was a single-item measure. We assert that an
organization’s financial health can affect the benefits ultimately obtained from an ERP-system
(Berchet and Habchi 2005; Hunton et al. 2003). We also feel that the recent financial crisis, which
occurred in the period of study, exacerbates the importance to include such a variable.
A similar, second regression will be run for organizations without an ERP-system, without the ‘go live’
variable.
The standardized residuals of these two regressions will subsequently be used to match
organizations. Thus, the residuals constitute our corrected size measure. Barber and Lyon (1996)
assert that when organizations are matched within the same 2-digit SIC-code, viable matches are
possible. As stated, this conforms to the correction for sectoral effects proposed by Hunton et al.
(2003). Of course, a perfect match in terms of standardized residuals will be highly unlikely.
Organizations, at best, will have roughly similar residuals. Barber and Lyon (1996) therefore decide to
use a 90%-110% range of the matching variable to carry out their matching. They say that as long as
the values of the matching variable are between 90%-110% of one another, a ‘good’ match can be
made. They also propose several ‘alternative rules’ to extend the possibilities of a match when the
abovementioned match cannot be made when using 2-digit SIC-codes. The following rule proved to
be the most promising in their study on organizational performance:
“First, we attempt to match … within the 90%-110% filter, using all firms in the same
one-digit SIC code. If we still find no … match, then we try to match … within the 90%-
110% filter using all firms without regard to SIC code. If we still find no … match, our
third step is to use the firm… closest to the firm in question, without regard to SIC
code” (Barber and Lyon, 1996, p370).
Following Hunton et al. (2003), wee will apply the same alternative rule here, even though
organizational size is our matching variable (instead of organizational performance).
The statistical test that in the Barber and Lyon (ibid.) study yielded the most statistical power is the
non-parametric Wilcoxon test (Hair et al. 2010). Its application led to consistently more powerful
3
For details, refer to Appendix I.
results than (in)dependent sample Student t-tests. This happens to be the same test used by Hunton
et al.(2003). It can accommodate very small sample sizes (with typically even less than ten
observations) and still have substantial statistical power (Noether 1987). Therefore, we will apply it as
well to test for differences in organizational benefits between the matched firms.
The six dimensions of organizational benefits distilled by Shang and Seddon (2002) are linked with
ten aspects in our survey (see Appendix I for details). Respondents were asked to indicate how they
judged the performance of their organization on these ten aspects, both currently and three years
ago. A Likert 1-7 scale was employed to assess this (Grabski and Leech 2007). A low score on this
scale indicates poor performance on a particular aspect, whereas a high score indicates very good
performance. Cronbach’s alpha will be used to substantiate the operationalization chosen.
3.2 Results
All ten aspects we used to operationalize Shang and Sheddon’s (2002) organizational benefits
concept could be grouped together. For the current data, Cronbach’s alpha was 0.770. For the data
from three years ago it was 0.836. Both values are larger than the commonly used threshold of 0.70
(Nunnally 1978).
A Cronbach’s alpha analysis was also performed to see whether the six items discerned by Vluggen
(2006) to capture PEU could be grouped together. Alas, Cronbach’s alpha was only 0.653. Although
below the 0.70 threshold, the value of 0.653 is deemed acceptable for exploratory studies, and was
therefore satisfactory for our purposes (Nunnally 1978). Deletion of items within the construct would
not have yielded a larger Cronbach’s alpha value.
In the regression analyses, multicollinearity was considered by checking pairwise Spearman rank
corrections for all explanatory variables. No problems were noted, for both regressions. The highest
bivariate correlation was between FINHEALTH and PEU for organizations with an ERP-system (-
0.183). No correlation was significant at the 5% level of significance.
By and large, the regression results of the two regressions for LOGMSALES were not impressive, as
can be seen in Table 1, panel A and B. In panel A it is shown that the regression for ERP-related
2
organizations had an R of 0.027 (with 21 observations), with no statistically significant variables at
either the 1%, 5% or 10% level of significance, except for the constant term. Both LOGMSALES and
the standardized residuals of the regression were normally distributed on the basis of a Kolmogorov-
Smirmov test at all of the aforementioned levels of significance, with p-values equal to 0.877 and
0.898 respectively (Hair et al. 2010). As shown in Table 1, panel B, roughly the same conclusions
held for the regression for the group of non-ERP adopting organizations (with 19 observations). This
regression yielded an R2 of 0.058, with only the constant term being significant at the1%, 5% and 10%
level of significance..P-values of the Kolmogorov-Smirov test for the LOGMSALES variable and the
standardized residuals were 0.612 and 0.543 respectively, suggesting normal distributions in both
cases. Visual inspection of predicted and standardized residuals suggested that homoscedasticity
was present in both regressions (see also Hair et al. ibid.).Two Glejser tests were carried out to
substantiate this conclusion (Gujarati and Porter2008). In this test, the absolute values of the
residuals of a regression become the dependent variable in a new regression, using all of the
explanatory variables of the original regression. Heteroscedasticity may be present when one or more
coefficients are significant in the new regression model. This was not the case in either of the two
regressions carried out here. This corroborated our visual impression that the residuals of both
regressions were homoscedastic.
Table 1: Regression results for LOGMSALES for organizations with (panel A) and without (panel B)
an ERP-system.
Panel A
F-statistic for
model (p-value):
.158 (0.923); R2:
.027
Panel B
F-statistic for
model (p-value):
.488 (0.622); R2:
.058
Table 2 depicts the sectors in which the 21 sample organizations with and 19 sample organizations
without an ERP-system operated, according to the respondents. This gives us some insight into the
number of viable matches that can be made to assess H1 and H2.
Table 2: Sample overview of ERP- and non-ERP adopting organizations per SIC-code
From Table 2, it is immediately clear that even when including 1-digit SIC-codes in our analysis, not
all organizations can be matched based solely on the sector in which they operate, for the mostly
dissimilar number of observations (standardized residuals) per sector (see also Barber and Lyon
1996).
For instance, there are five organizations in the dataset with SIC-code 28 (producers of metal
products) among the organizations with an ERP-system, but only two among the non-adopting
organizations. Therefore, at best, two organizations can be matched for this particular sector at the 2-
digit SIC-code level. Even when the possibilities for matching are extended following Barber and
Lyon’s alternative rule, we can only viably match 15 of the 19 organizations that could in principle
have been matched, as we deem the difference between standardized residuals to be too large for
the remaining four organizations (differences were more than 100%, and in two cases even more than
300 and 1000%). This we find a shortcoming of Barber and Lyon’s alternative rule: it encourages
researchers to keep on matching cases until there are none left in the database. In the end, this may
mean linking too dissimilar organizations, but still calling such links a ‘match’. This we believe is
misleading, and is something we try to circumvent here.
Two Wilcoxon paired-sample tests were conducted to assess H1 and H2. The results of these tests
have been summarized in Table 3. We could not reject H1 at either the 5% or 10% level of
significance, but we could at the 1% level of significance. H2 could not be reject at either the 1%, 5%
or 10% level of significance (p-values were 0.030 and 0.670 respectively). Thus, before the
implementation of an ERP-system, organizational benefits were not significantly larger for
organizations that implemented such a system than for organizations that did not implement or have
such systems, but there is some indication that ERP-adopting organizations outperformed non-
adopting organizations after the implementation (and within three years), even though the latter result
is not particularly strong, judging from the corresponding p-value of 0.030. We believe that the
number of observations in our sample did not affect this outcome, as Wilcoxon tests have been
shown to have sufficient power in case of small samples (Barber and Lyon 1996;Noether 1987).
Table 3: Wilcoxon test results for H1 and H2.
ERP-adopting
Difference in .670
organizations
organizational
(2007): 4.353
benefits of matched
(4.500)
organizations (2007)
Non-adopting
organizations
(2009): 4.673
(4.500)
Non-adopting
organizations
(2007): 4.220
(4.200)
In the following section, we will discuss our results against the background of the
extant literature.
4. Discussion and conclusions
Our results partly corroborate the results in the ERP-literature, but they contradict others. For
instance, all of the SMEs that implemented an ERP-system in the last three years needed less than
three years for organizational benefits to start materializing. This is in line with Nicolaou (2004) and
Nicolaou and Bhattacharya (2006), who suggested a period of two years for this to happen, but not
with Kallunki et al. (2011) and Liu et al. (2008), who found a period of at least three years more likely.
However, the aforementioned studies do not focus on SMEs. Perhaps the relatively less intricate
organizational structure of many SMEs as compared to large multinational companies may be a
reason for this different outcome. However, the fact that ERP-adopting organizations tend to
outperform non-adopting organizations, sometime after a system’s adaptation, and perhaps only for a
limited period of time, is something that has been mentioned in the ERP-literature before (Bernroider
2008; Davenport 1998; Sneller 2010).
One might wonder if the ERP-adopting and non-adopting firms in our sample underwent statistically
significant increases or decreases in organizational benefits over the last three years themselves.
This could provide additional information on our likely acceptance of H1 and rejection of H2.Two other
Wilcoxon paired-sample tests were carried out to assess this, using our entire sample of 21 ERP-
adopting and 19 non-ERP adopting firms. As indicated in Table 4, in both cases, we rejected the null
hypothesis that no changes in organizational benefits occurred in the 2007-2009 period at the 1%, 5%
and 10% level of significance (p-values was 0.001for both tests). Thus, by and large, both ERP-
adopting and non-adopting firms had significant changes in organizational benefits in three years’
4
time - the changes being positive in both cases . Although organizational benefits were slightly less
for non-ERP adopting firms than for adopting firms both three years ago and now, they increased by
almost the same amount in the intervening period. Still, changes were sufficiently different for H1 to
be accepted. This suggests that other factors affected both adopting and non-adopting organizations’
performance between 2007-2009 considerably, rather than or in addition to just the implementation of
an ERP-system. Although such systems ostensibly had some impact on the performance of adopting
firms vis-à-vis non-adopting firms, other factors must have influenced our results as well, judging from
the latter two tests. Thinking that ERP-implementations are the main culprit of this result may thus be
too far-fetched.
Table 4: Wilcoxon test results for additional analyses.
4
As can be seen in Table 4, for all ERP-adopting firms, the mean (median) level of organizational benefits rose from 4.286
(4.200) to 4.871 (4.800) while for non-adopting firms it rose from 4.316 (4.200) to 4.853 (4.700).
5
Note that in Table 3, mean and median organizational benefits for the 15 matched organizations are presented, whereas in
Table 4, all ERP and non-ERP adopting organizations in our sample are taken along.
(4.200)
Acknowledging that there are many factors influencing an organization’s performance apart from an
ERP-system, the question arises how these factors might be disentangled to assess their individual
impact. Unfortunately, this may be extremely difficult, no matter which theoretical frameworks are
adopted or research methods are applied. Think for example of the effects of management team
changes on organizational performance, or the empathy that is felt for a consultant who helped an
organization during an ERP-implementation, which makes that some employees continue to put in
extra hours to make the system a success. How can such effects be set apart from the effects caused
by an ERP-system by itself, when examining an organization’s day-to-day business? All of these
effects are obviously interrelated, often intricately so (Scapens 2006). On top of this, organizations
can be seen as constellations of human and non-human actors, each with a specific past, and with
their own set of rules and routines that have developed over time, which may be difficult to control and
change (Boudreau and Robey 2005). How can such organizations be compared with others, so that
reliable, general statements about the effects of ERP-systems on organizational conduct can be
made (see also Argyris 1964)? Many of these aspects are overlooked when organizations are treated
as ‘black boxes’, as is commonly done in much of the ERP-related literature on financial performance
effects (see also Grabski et al. 2011). Some claim that the fact that this has not often been done has
resulted in a highly influential ERP-rhetoric: ERP-systems are supposed to be beneficial for
organizations when they are implemented ‘correctly’ and are used throughout an organization without
difficulty. This is what these systems ‘simply’ are made to do, given their focus on ‘best practices’
(Lang and Wagner 2006).
Nevertheless, we assert that there is clearly added value in this paper. It lies in the fact that as of yet,
not much empirically supported research seems to be available for Dutch SMEs (Bernroider and Tang
2003). We admit that generalizable results cannot be claimed on the basis of this study. Still, valuable
insights into some of the effects that ERP-implementations might (not) realize in specific settings
when looking at non-financial performance benefits have been generated. Further research, using
larger samples, and also, perhaps, taking into account other factors like the presence of specific
integrating mechanisms that might help in realizing the alleged integration benefits of ERP-systems
(Grabski and Leech 2007; Kallunki et al. 2011; Kumar et al. 2002), could yield further important
insights into an, in our view, under-researched area in the ERP-literature.
References
Adam, F. and O’Doherty, P. (2003). “ERP projects: good or bad for SME’s?” In: Shanks, G., Seddon, P. and
Willcocks, L. (Eds.), Second-wave enterprise resource planning; implementing for effectiveness,
Cambridge: Cambridge University Press, pp275-298.
Ahrens, T. and Chapman, C.S. (2007). “Management accounting as practice”. Accounting, Organizations and
Society, Vol.32, No. 1, pp1-27.
Al-Mashari, M., Zairi, M. and Okazawa, K. (2006). "Enterprise resource planning (ERP) implementation: a useful
road map". International Journal of Management and Enterprise Development, Vol. 3, No.1/2, pp169-180.
Anthony, R.N. (1965). Planning and control systems: a framework for analysis. Cambridge: Harvard Business
School Publications.
Argyris, C. (1964). Integrating the individual and the organization. New York: John Wiley& Sons.
Barber, B.M. and Lyon, J.D. (1996). “Detecting abnormal operating performance: the empirical power and
specification of test statistics”. Journal of Finance & Economics, Vol. 41, No. 3, pp359-399.
Berchet, C. and Habchi, G. (2005). “The implementation and deployment of an ERP system: an industrial case
study”. Computers in Industry, Vol. 56, No. 6, pp588-605.
Bernroider, E.W.N. (2008). “IT governance for enterprise resource planning supported by the DeLone–McLean
model of information systems success”. Information & Management, Vol. 48, No. 5, pp257-269.
___ and Tang, N (2003). "Einführungsentscheidung, Bewertung und Auswahl von
betriebswirtschaftlicherStandardsoftware (ERP-Systemen) in KMUs",
DiskussionspapierezumTätigkeitsfeldInformationsverarbeitung und Informationswirtschaft (Working Papers
on Information Processing and Information Management, No. 03/2003, Wirtschaftsuniversität Wien, Vienna,
Austria.
Botta-Genoulaz, V. and Millet, P-A. (2005). “A classification for better use of ERP systems”. Computers in
Industry, Vol. 56, No. 6, pp573-587.
Boudreau, M.C. and Robey, D (2005). “Enacting integrated information technology: a human agency
perspective”. OrganizationScience, Vol. 16, No. 1, pp3-18.
Bradley, J. (2008). “Management based critical success factors in the implementation of enterprise resource
planning systems”. International Journal of Accounting Information Systems, Vol. 8, No. 3, pp175-200.
Buckhout, S., Frey, E., and Nemec, Jr., J. (1999). “Making ERP succeed: turning fear into promise”. Journal of
Strategy and Business, Vol.15, No. 1, pp60-72.
Davenport, T.H. (1998). "Putting the enterprise into the enterprise system". Harvard Business Review, Vol. 76,
No. 4, pp121-131 (July/August).
Eckartz, S., Daneva, M., Wieringa R., van Hillegersberg, J. (2009). “A conceptual framework for ERP benefit
classification: a literature review”. Technical Report TR-CTIT-09-04, Centre for Telematics and Information
Technology, University of Twente, Enschede, the Netherlands (ISSN 1381-3625).
Esteves, J. and Bohorquez, V. (2007). “An updated ERP systems annotated bibliography: 2001–2005”.
Communications of the Association for Information Systems, Vol. 19, Art. 18, pp386-446.
Grabski, S.V. and Leech, S.A. (2007). “Complementary controls and ERP implementation success”. International
Journal of Accounting Information Systems, Vol. 8, No. 1, pp17-39.
Grabski, S.V., Leech, S.A. and Schmidt, P.J. (2011).“A review of ERP research: afuture agenda for accounting
information systems”.Journal of Information Systems, Vol. 25, No. 1, pp37-78.
Gujarati, D. and Porter, D. (2008).Basic econometrics(5thedition).New York: McGraw-Hill/Irwin.
Hair, J.F., Black, W.C., Babin, B.J. and Anderson, R.E. (2010). Multivariate data analysis; a global perspective
(7th edition). New Jersey: Prentice Hall.
Hong, K. and Kim, Y. (2002). “The critical success factors for ERP implementation: an organizational fit
perspective”. Information & Management, Vol. 40, No. 1, pp25-40.
Hopwood, A.G. (1987).“The archaeology of accounting systems”.Accounting,Organizations and Society, Vol. 12,
No. 3, pp207-234.
Hunton, J., Lippincott, B. and Reck, J. (2003). “Enterprise resource planning systems: comparing firm
performance of adopters and nonadopters”. International Journal of Accounting Information Systems, Vol. 4,
No. 3, pp165-184.
Jones, M.C., Cline, M. and Ryan, S. (2006). “Exploring knowledge sharing in ERP implementation: an
organizational culture framework”. Decision Support Systems, Vol. 41, No. 2, pp411−434.
Kallunki, J-P., Laitinen, E.K. and Silvola, H. (2011). “Impact of enterprise resource planning systems on
management control systems and firm performance”. International Journal of Accounting Information
Systems, Vol. 12, No. 1, pp20-39.
Kaplan, R.S., and Norton, D.P. (1993). "Putting the balanced scorecard to work". Harvard Business Review, Vol.
71, No. 5, pp134-147.
Keller, E.L. (1999). “Lessons learned”. Manufacturing Systems, Vol. 17 No. 11, pp44-50.
Kumar K. and van Hillegersberg, J. (2000). “Enterprise resource planning experiences and evolution”.
Communications of the ACM, Vol. 43, No. 3, pp22-26.
Maheshwari, B. and Kumar, U. (2002). “ERP systems implementation: best practices in Canadian government
organizations”. Government Information Quarterly, Vol. 19, No. 2, pp147-172.
Light, B. and Wagner, E.L. (2006). "Integration in ERP environments: rhetoric, realities andorganisational
possibilities". New Technology, Work and Employment, Vol. 21, No. 3, pp215-228.
Liu, L., Miao, R. and Li, C. (2008). “The impacts of enterprise resource planning systems on firm performance: an
empirical analysis of Chinese chemical firms”. IFIP International Federation for Information Processing, Vol.
254/2008, pp579-587.
Markus, M.L. and Tanis, C. (2000). “The enterprise systems experience – from adoption to success”. In: Zmud,
R.W. (Ed.), Framing the domains of IT research: glimpsing the future through the past, Cincinnati: Pinnaflex
Educational Resources, pp173-207.
Nicolaou, A.I. (2004). “Firm performance effects in relation to the implementation and use of enterprise resource
planning systems”. Journal of Information Systems, Vol.18, No. 2, pp79-105.
___ and Bhattacharya, S. (2006). “Organizational performance effects of ERP systems usage: the impact of post-
implementation changes”. International Journal of Accounting Information Systems, Vol. 7 No. 1, pp18-35.
Noether, G.E. (1987). “Sample size determination for some common nonparametric tests”. Journal of the
American Statistical Association, Vol. 82, No. 398, pp645-647.
nd
Nunnally, J.C. (1978). Psychometric theory (2 edition). New York: McGraw-Hill.
Pairat, R. and Jungthirapanich, C. (2005). “A chronological review of ERP research: an analysis of ERP
inception, evolution, and direction”. Proceedings of the IEEE International Engineering Management
Conference, Vol. 1, pp288-292.
Poston, R. and Grabski, S. (2001). “Financial impacts of enterprise resource planning implementations”.
International Journal of Accounting Information Systems, Vol. 2, No. 4, pp271-294.
Ross, J.W. and Vitale, M.R. (2000). “The ERP revolution: surviving vs. thriving”. Information System Frontiers,
Vol. 2 No. 2, pp233 -241.
Scapens, R.W. (2006). “Understanding management accounting practices: a personal journey”. British
Accounting Review, Vol. 38, No. 1, pp1-30.
Scott, J. (1999). The FoxMeyer Drugs’ bankruptcy: was it a failure of ERP? In: Leitheiser, R. (Ed.), Proceedings
of the fifth American conference on information systems. Milwaukee: Association for Information Systems,
pp223-225.
Shang, S. and Seddon, P. (2002). “Assessing and managing the benefits of enterprise systems: the business
manager’s perspective”. Information Systems Journal, Vol. 12, No. 4, pp271-299.
Sneller, L. (2010). Does ERP add company value? A study for the Netherlands and the United
Kingdom.Alblasserdam: Drukkerij HAVEKA.
Soja, P. and Paliwoda-P kosz, G. (2009). “What are real problems in enterprise system adoption?”. Industrial
Management & Data Systems, Vol. 109, No. 5, pp610-627.
Stefanou, C.J. (2001). "A framework for the ex-ante evaluation of ERP software". European Journal of
Information Systems, Vol. 10, No. 4, pp204-215.
Vluggen, M. (2006). Enterprise resource planning systems: an empirical study of adoption and effects. PhD
dissertation, Maastricht: Universiteit Maastricht.
Wah, L. (2000). “Give ERP a chance”. Management Review, Vol. 89, No. 3, pp20-24.
(1= very poorly; 2= quite poorly; 3= somewhat below expectations; 4= as expected; 5= somewhat
better than expected; 6= quite well; 7= very well; 8= don’t know)
1 2 3 4 5 6 7 8
1. Internal communication
2. Standardization of work processes
3. Quality of internal reports
4. The degree to which work activities and decision
power have been relegated to other employees than
managers
5. Flexibility of work processes
6. The size of budgets available for internal and
external courses
7. Building a common, organization-wide vision
8. Focus on core tasks
9. Employee satisfaction
10. Mutual behaviour of employees
Financial health [Berchet and Habchi 2005; Hunton et al. 2003]
How far do you agree with the following statement: “I work in a financially healthy organization”?
6
This question was repeated to assess the situation in an organization three years ago.
Below you can find several statements that are presented as opposite pairs. All statements concern
the market(s) and environment(s) your organization operates in. Could you please tell us how far
these statements currently reflect the situation your organization faces?
don’t
1 2 3 4 5 6
the marketing know
the marketing strategy
strategy of our
of our organization is
1. organization is
hardly ever changed to
changed very often to
fulfill customer needs
fulfill customer needs
don’t
our line of products 1 2 3 4 5 6
our line of products know
and services stays very
2. and services changes
much the same over
very quickly over time
time
don’t
1 2 3 4 5 6
it is relatively easy to know
our competitors are
3. guess what our
highly unpredictable
competitors are doing
don’t
1 2 3 4 5 6
it is relatively easy to it is extremely difficult know
4. determine what our to determine what our
customers want customers want
don’t
1 2 3 4 5 6
we continuously have we hardly ever know
5. to anticipate changes have to anticipate
in our environment in changes in our
order for our environment in
organization to survive order for our
organization to
survive
don’t
1 2 3 4 5 6
our production know
6. our production technology changes
. technology is stable very rapidly
Keyword: Enterprise resource planning system, Effective use, Delphi methodology, Process
1. Introduction
Many large and small-scale process industries in both developed and developing economies have
invested in Enterprise Resource Planning (ERP) systems (AberdeenGroup, 2008). The adoption of
such systems replaces outdated legacy systems in order to meet the ever-changing business
environment (Esteves, 2009; Hsieh and Wang, 2007). One significant reason for the adoption of ERP
systems by these organisations is to account for the standardized process and integration of the
business unit functions for decision-making (Hakkinen and Hilmola, 2008). Obtaining inter-
organizational value chains and transactional backbone for e-business collaboration with clients,
partners and suppliers have also been major drivers of ERP wide adoption by firms (Jasperson et al.,
2005, Wang et al., 2008).
There have been discussions in much of the extant literature on ERP implementation failures (Yu,
2005). However, it continues to be the most widely adopted information systems (IS) globally (Esteves
and Bohorquez, 2007). Several studies have shown that the enormous investments in ERP systems
could only achieve transactional efficiency, leaving the emergent strategic value of the system under-
utilized (Chen, 2001; Hsieh and Wang, 2007; MacKinnon et al., 2008). Though the benefits of such
systems are enormous, users frequently take advantage of only the most basic use, instead of
effectively using the system to create business values (Sousa and Goodhue, 2003). The effective use
of ERP systems is not just the mandatory basic use (Hsieh and Wang, 2007; Sousa and Goodhue,
2003). Rather, it is the exploratory and exploitative use of the features of the application to enhance
organisations’ productivity (Boudreau, 2002; Loftus, 2008; Sousa and Goodhue, 2003). Effective use
is to get users to move beyond the routine use demanded by their assigned task(s) in the workplace
and uncover new ways of using the system features (Sousa and Goodhue, 2003).
To effectively utilize ERP systems in the process related industries (such as Chemical, Food &
Beverage, Pharmaceuticals, Metals & Mining, Pulp and Paper Industries) has been a lingering
problem (Aarde, 2007; AberdeenGroup, 2008; Hameri and Lehtonen, 2001; Lail, 1999; Shah, 2005).
Although deployment of ERP systems originated in the discrete industries, process industries are
increasingly adopting such systems to support their cyclical business operations (Aarde, 2007; Lail,
1999). Due to the vertical structure in the organic growth of this type of industry, ERP systems have
replaced their legacy sets of IS to support supply chain operations (Hameri and Lehtonen, 2001; Lail,
1999). The supply chain operation stems from optimal utilization of the existing plant capacity through
operation planning and scheduling (Thompson, 2000). Such planning and scheduling demand
effective use of ERP systems in mass customization, not in the products, for the industry operates low
volume functional products, but in the services associated with the products (Akkermans et al., 2003).
The functional products characterised by a high degree of variability distinguish the process industries
from the discrete industries (Aarde, 2007). Usability problems in ERP have been attributed to poor
business performance and low managers’ productivity in these types of industries (Iansiti, 2007; Lail,
1999; Shah, 2005). Although ineffective use is among the major challenges confronting ERP post-
implementation success, little research has been conducted on this issue, especially in the process
industry (Hamerman, 2007; Hsieh and Wang, 2007).
Often, scholars have observed that individual usage of ERP systems is impacted by social,
organizational, environment and individual adoption factors (e.g., Al-Mashari et al., 2003; Amoako-
Gyampah, 2007; Chang et al., 2008; Hakkinen and Hilmola, 2008; Hsieh and Wang, 2007; Ifinedo,
2008; Nicolaou, 2004). Most of these factors, which cut across the implementation and post-
implementation phases of ERP life cycle, are labelled as enablers of the system usage (Cenfetelli,
2010). However, there exist inhibitors that serve solely to discourage effective system usage and are
distinctly different from enablers of such usage (Cenfetelli, 2010). Cenfetelli (2010) defined inhibitors
as those negative factors that discourage IT usage (in this case ERP systems) when present, but do
not necessarily favour such usage when absent. This asymmetric effect implies that inhibitors are not
necessarily the mirror opposite of enablers, but are qualitatively distinct constructs that are
independent of but may coexist with enablers (Cenfetelli, 2010; Bhattacherjee and Hikmet, 2007). To
focus on such inhibitors, which have received less attention from prior studies, requires the views of
experienced users of ERP systems. The perceptions of such users will assist in establishing the link
between the inhibitors and the effective use of ERP systems as well as in ranking them for managerial
attention. The study tends to focus on the process industry because a recent study by Zhu et al.
(2010) notes that the relative effects of the factors on the post-implementation success of ERP differ
across different industries. Hence, in order to obtain a broader view of these inhibitors, Technology-
Organization-Environment (TOE) framework, Task Technology Fit (TTF) and the theory of usage
inhibition by Cenfetelli (2004) are used as lenses to guide the study. The purpose of integrating these
theories is to offer a suitable framework for the categorization of the inhibitors before subjecting them
to empirical testing. Consequently, two research questions are incorporated to focus the investigation:
1) What factors do managers in the process industry consider to be inhibitors to the effective use
of ERP systems?
2) Which of the factors / inhibitors should managers in the process industry consider more
deserving of their attention?
The study is particularly interested with the strategic managers since they are the ones to drive the
effective use of the system (Ifinedo, 2008; Loftus, 2008). A Delphi approach is adopted to
systematically identify and rank the major inhibitors. The Delphi technique further grants the study the
opportunity to engage multiple perspectives of ERP key users considered as experts in the use of the
systems from New Zealand’s process industries.
The rest of this paper is organized as follows: Section 2 consists of the literature review that focuses
on the need to re-conceptualize ERP usage and the theoretical background with relevant literature on
ERP usage. Section 3 describes the Delphi research methodology and the findings. Section 4
discusses the findings and presents the implications of these findings to both research and practice,
the limitations of the study, and future research directions. Section 5 concludes this paper.
2. Literature review
implementation as an all-inclusive process spanning this four-phase model. However, treating ERP
implementation as an all-inclusive process for whatever reason(s) limits the crucial and holistic
understanding of each of the phases of the ERP life cycle. To avoid such will help in the proactive
seeking of relevant solutions for achieving success at each specific phase. Without diagnosing every
phase of the ERP innovation process to understand the weakest link, we argue that the overall ERP
success will be conceptualised vaguely.
The realisation of ERP business values has often been assumed to be met when implementation
success is achieved. In many cases, however, such an assumption has proven to be incorrect over
time (Markus and Tanis, 2000; Soja and Paliwoda-Pekosz, 2009; Yu, 2005; Zhu et al., 2010). In fact,
ERP failure cuts across the four phases of the ERP life cycle, with significant and enduring negative
consequences in the appropriation phase when the enormous resources expended in the installed
systems did not yield an appreciable business value due to underutilization (Zhu et al., 2010). This
brings us to the urgent need to understand the concept of “use” in the ERP research domain and, on
the basis of this understanding, will the inhibitors of effective use be identified and empirically
validated.
Being often considered as an obscure territory, the concept of “use / usage” and its ambiguous
application in the ERP research domain has been contentious (Brown et al., 2002; Nah et al., 2004).
The ambiguity associated with this concept (use), stems from researchers equating ERP systems to
any other IS (Ifinedo, 2008). ERP systems are different from traditional IS due to their ability to impact
significantly on the “…technological, operational, managerial, strategic, and organizational
components of the adopting firm” (Ifinedo, 2008, p.552). It is mandatory for users to utilise the
installed ERP system as it reorganises an adopting firm to fit into its default business logics, popularly
expressed as “best practise” (Yu, 2005). Its nature of sophistication demands that firms solicit external
consultants to assist in developing and transferring their ERP knowledge skills to the adopting firms
(Ifinedo, 2008; Soja and Paliwoda-Pekosz, 2009). Even prior to implementation, it is expedient for
firms to understand how to effectively use the system to achieve their organisational goals (Ifinedo,
2008). A lack of appreciation of the uniqueness of ERP can cause the demise of the adopting
organisation, after having expended enormous resources (Chen, 2001; Ifinedo, 2008; Yu, 2005).
Consequently, it is evident that frequent application of the concept use to measure ERP success,
demands re-conceptualization to reflect its uniqueness and specific design features that need not to
be generalized to other IT artifacts (Gable et al., 2003; Nah et al., 2004). To do this depends on the
accurate understanding of the mandatory and the non-mandatory nature of adopting ERP systems.
Therefore, it is inappropriate due to bias to employ the construct “use / usage“, “user acceptance” or
“intention to use” to evaluate ERP, since users are already obligated to use the application in their
everyday tasks. This study argues that these constructs (i.e. “use”, “user acceptance” or “intention to
use”) that stem from attitude-intention-behaviour theories are inconsequential and highly prone to
bias, since it is obvious that employees are mandated to use such systems in their organisation,
especially during the early individual’s adoption to get his or her job going. Extant literature (e.g.,
Brown et al., 2002; Gable et al., 2003; Nah et al., 2004) has also criticised employing these constructs
to assess a mandatory phase of ERP success. However, as users gather experience with the system,
individual usage of ERP shifts to the non-mandatory phase driven by the users’ intrinsic motivation to
learn more of the system capabilities. Therefore, a more robust and parsimonious construct termed
“effective use” or “extended use” could be employed to assess ERP systems (Boudreau, 2002; Hsieh
and Wang, 2007; Loftus, 2008; Sousa and Goodhue, 2003). Sousa and Goodhue (2003) note that the
effective use of an ERP system is discretionary and significantly driven by the users’ business
knowledge, the conceptual knowledge of the application and its procedural knowledge and motivation.
Paraphrasing Sousa and Goodhue (2003), exploratory and exploitative use of the system cannot be
mandated; despite not being part of the formal job expectations per se, it contributes significantly to
organisational productivity. Boudreau (2002) noted the erroneous view of assuming “use” to mean
“effective use” as he observes that the degree of use does not necessarily translate to effectiveness.
Boudreau classifies “effective use” to mean “quality of use”, which is an exploitative use of the
functionalities of the system for organisations’ competitive advantage (Boudreau, 2002; Loftus, 2008;
Sousa and Goodhue, 2003).
A clearer understanding of this concept of “use”, especially in the ERP environment, can also be
derived from the earlier work of Lassila and Brancheau (1999). Lassila and Brancheau (1999)
classified “use” in the context of organisational environment to have four “equilibrium states” depicting
the increasing sophistication of use in the ERP environment. These four states pictured: (1) limited
“use” (i.e. low-integration), (2) “use” as to support existing processes (i.e. standard adoption / basic
use), (3) “use” as to redesign existing work processes (i.e. expanding), and (4) “use” as to exploit the
capabilities of ERP to align with the ever changing business environment (i.e. high integration)
(Boudreau, 2002). Therefore, the effective use of ERP revolves around ‘expanding’ to ‘high
integration’ where users have gone beyond the mandatory use (i.e. low-integration and standard
adoption) to a discretionary exploitation of the system for competitive advantage (Loftus, 2008; Sousa
and Goodhue, 2003).
Bringing this concept of inhibitors theory to the ERP research domain, ERP default complexities
associated with system features, such as difficulty in accessing the correct functionality, system
output limitations, poor support in error situations and complexity of the system, have been attributed
to ERP usage inhibitors (e.g., Hamerman, 2007; Iansiti, 2007; Topi et al., 2005; Soja and Paliwoda-
Pekosz, 2009; Van Everdingen, 2000). Sousa and Goodhue (2003) conceptualize the effective use of
ERP to be influenced by employees’ understanding of the system’s procedural knowledge (i.e. syntax,
semantics and commands), application knowledge (the understanding of the business processes’
workflow mapped in the application) and business context knowledge (the understanding of the
processes specific to the business across the organizational functions). However, poor acquisition of
such knowledge, due to low absorptive capacity and inadequate training to acquire such knowledge
and systems inflexibility, constitute the major inhibitors of ERP system usage (Al-Mashari et al., 2003;
Iansiti, 2007; Kouki et al., 2006; Topi et al., 2006; Soja and Paliwoda-Pekosz, 2009; Sousa and
Goodhue, 2003). Drawing from Task-Technology Fit (TTF) theory, a system misfit to the employees’
assigned tasks / business process has been noted to impact negatively on ERP individual
performance (Kositanurit et al., 2006; Soja and Paliwoda-Pekosz, 2009; Somers and Nelson, 2003;
Smyth, 2001). Lack of continuous improvement strategy from the top management, poor vendors’
service support, poor communication and lack of motivation among the employees are also
considered as inhibitors to ERP system usage (Al-Mashari et al., 2003; Chang et al., 2008; Soja and
Paliwoda-Pekosz 2009).
To categorize the enablers from prior studies, the TOE framework (Tornatzky and Fleischer, 1990)
has been noted to be commonly used (Pan and Jang, 2008; Zhu et al., 2010). As generic theory of
technology diffusion, the TOE framework can as well be used for studying related factors that
dissuade ERP usage adoption. The TOE framework identifies three facets of the IS adopting
organisation’s contexts, namely the technological, organisational and environmental context. The
technological context describes both the internal and external technologies relevant to the
organisation. The organizational context consists of descriptive measures such as a firm’s size and
scope, centralization, formalization, and complexity of managerial structure, the quality of human
resources, and the amount of internal slack resources. The environmental context is the arena in
which a firm conducts its business: its industry, competitors and dealings with government (Tornatzky
and Fleischer, 1990). Prior studies of ERP adoption drew from TOE to elicit enabling factors that
positively impact on ERP success or performance (e.g., Pan and Jang, 2008; Shahawai and Idrus,
2010; Zhu et al., 2010). Consequently, building on the dual-factor model, TOE can as well be used to
elicit inhibitors of effective use of ERP systems, since IS usage is determined by a simultaneous
examination of enabling and inhibiting factors (Cenfetelli, 2004).
3. Methodology
For 11 months (May 2008 – April 2009), the panel members assessed the content standards and
indicators that make up the three rounds stated earlier. To improve the content standards of the
issues under investigation in each round, the panel members were asked for suggestions. After each
round, the standards were revised based on their judgments and comments after analyses by the
researcher then sent to the panel members as feedbacks.
The transcribed texts were imported into Nvivo qualitative software for content analysis and coding to
elicit categories. First, we labelled the dimensions of the categories according to the “meaning they
evoke”, “words of respondents” or “borrowed from literature” (Strauss and Corbin, 1998). The main
categories consist of non-repetitive, non-overlapping statements, which were identified as the
meaning units of experience. In the next step, we went line-by-line through the texts in which we
assigned each text phrase to a category. If a text phrase did not match the established categories, we
defined a new category (inductive categorization). The whole system of categories was revised and
adapted with regard to the subject and aims of the qualitative Delphi approach (i.e. eliciting inhibitors
to the effective use of ERP systems from the experts) before finalizing the analysis. The resulting list
of categories reflected the possible inhibitors. The overall categorization was done by the researcher
and assisted by an independent coder; any differences in opinions were resolved by discussion. The
categorization items include inadequate ERP expertise, ERP default attributes, non-collaborative
training among employees, low absorptive capacity, poor user involvement, lack of continuous
improvement, system misfit, consultant’s ineffectiveness, and poor vendors’ support. Drawing from
TOE framework, TTF theory, the dual factor model and the findings from the qualitative content
analysis in round one, a priori conceptual framework for the research was developed as shown in
Figure 2. This framework is a conceptual taxonomy of inhibitors identified for further investigation in
rounds two and three of the Delphi. It is meant to be representative, not causal; hence, there are no
arrows.
The three ranking round results are shown in Table 3. Round 3 of the mean ranking, where Kendall’s
W is (0.76 < p 0.05), indicates a strong agreement among the 18 panel members (Schmidt et al.,
2001).
Table 3: The three round Ranking of Inhibitors (n=18)
To ensure the practical relevancy of this research we synthesised the empirically developed list of 7
inhibitors of effective ERP usage, as shown in Table 3, with what was gleaned from the comments the
panellists made in round 3. As noted earlier, the comments emanate from the panellists’ answers to
the question on how such barriers reflect their companies’ situations and the suggested ways of
resolving them. From the developed items, only non-collaborative training among employees that is
within the organisational context is considered as a critical inhibitor and most deserving of managers’
attention, as shown in Table 3. From those three important inhibitors, only two (low absorptive
capacity and system misfit) are related to organisational issues, while one (inadequate ERP expertise)
is related to technical issues. Among the three moderate inhibitors, one (ERP default attributes)
belongs to technical issues, while lack of continuous improvement is of organisational issues and the
last inhibitor (poor vendors’ support) falls within environmental issues.
We first focus on training issues since it is the most prevalent inhibitor to effective use of ERP
systems. In the area of training, the implications are that process industries in New Zealand are
experiencing misfit in their training process. It is incorrect to envisage that the strategic users are not
trained; rather there are misalignments in the training process according to panel members.
Management should, in the future, engage consultant trainers that will work in collaboration with the
active users to develop their unique business context related training. The research strongly
supported the views of other scholars (e.g., Al-Mashari et al., 2003; Soja and Paliwoda-Pekosz, 2009;
Sousa and Goodhue, 2003) that users have to be trained. However, the findings from the study
suggests that users should be trained on the appropriate knowledge levels i.e. procedural knowledge,
application conceptual knowledge, and business context knowledge (Sousa and Goodhue, 2003), in a
participatory manner among the employees themselves. In doing this, the employees therein will
develop a healthy psychological ownership of the system, and to personally identify innovative users
to share their experience (Soja and Paliwoda-Pekosz, 2009; Sousa and Goodhue, 2003). In this way,
management could reduce the resources expended in organizational learning in the future. The
resources could be channelled to innovativeness by tapping into the users’ unique insight of the
business through effective use of the systems (Gupta, 2000; Sousa and Goodhue, 2003).
Second, low absorptive capacity among the employees, being defined by Cohen and Levinthal (1990)
as the firm's inability to appreciate an innovation, to assimilate and to apply it to new ends, strongly
requires organisational prior relevant knowledge and its investments in acquiring new knowledge
(Ravichandran, 2005). ERP, being a complex system, imposes a heavy learning curve and, as such,
the more a firm possesses prior ERP related knowledge, the more effective is its usage. ERP’s low
absorptive capacity and its default attributes of being complex as well as the perceptions it generated
among employees gave rise to inadequate ERP expertise. To lessen the effect, the right type of
training and individual skill development is highly needed (Calisir and Calisir, 2004; Soja and
Paliwoda-Pekosz, 2009).
Third, the presence of system misfit, where the system is not aligning with the business process,
contributed to ineffective system usage (Soja and Paliwoda-Pekosz, 2009). A typical example is in the
area of process industry supply chain (SC) operations. ERP’s support in SCM is of two dimensions:
operational and strategic (Akkermans et al., 2003). For operational, panellists are of the view that the
systems is compatible enough to support SC transactional functionality. However, in supply chain
design, the panellists emphasised the inflexibility of ERP to support SC design and knowledge
knowhow. For example, channel differentiation is used to service different market segments in the
process industry, but their present ERP is not flexible enough to support such without much
customization. Customization on its own is done with every necessary caution to avoid too much cost
in keeping the consultants on site and to avoid subsequent upgrade difficulties (MacKinnon et al.,
2008). For these reasons, the industry did not consider ERP as a critical enabler of Supply Chain, the
panel members commented. One key area vendors normally advice is redesigning the industry
business process to fit the functionality of the ERP systems to lessen customization (Soja and
Paliwoda-Pekosz, 2009). However, this does not encourage competitive advantage as everybody is
doing the same thing overtime. Another long-term issue the panellists revealed is the reengineering
initiatives in ERP systems to support the industry. The panellists highlighted that such initiative
demands a strategic leap that needs top management support (Ifinedo, 2008). It also demands
external mediating agents such as the local vendors / consultants whose services are noted to be
inefficient, probably due to inexperience, and the cost of hiring tested consultants outside New
Zealand is usually prohibitive.
Fourth, the inadequate ERP expertise is heightened by the complex default attributes of ERP systems
and lack of continuous improvement. Default attributes of ERP systems as an inhibitor stems from
employee’s difficulty in accessing the correct functionality to support their work-related tasks as well
as the system’s poor support in error situations. Based on this, the employees tend to lack expertise
in handling the ERP system to achieve their dynamic and strategic job roles in the New Zealand
process industry.
Fifth, a lack of continuous improvement by top management in not upgrading the ERP system
heightens the ineffective use of ERP systems. Several ERP vendors in their recent updates tend to
address users’ complexity issues, improve on the system’s flexibility and its business process
adaptability. However, if the adopting organisations are not appropriating the upgrades on time they
will continue to face usability challenges. This situation was buttressed by one panellist’s comment
that “they now have a legacy ERP system”, suggesting that after the implementation the management
shows less concern in upgrading the system. Top management on their own are not committed to
upgrades due to the risk associated with it and the huge financial implications (Soja and Paliwoda-
Pekosz, 2009; Van Everdingen, 2000).
Lastly, poor vendors’ support also contributes to ineffective use of ERP by strategic managers. ERP
vendors tend to be proactive and very much concerned with implementation success. However, once
the system “goes live” they seem to relax in their pragmatic approach to assist the adopting
organisations to ensure effective use of the system (Hakkinen & Hilmola, 2008; Pan et al., 2007). The
adopting organisation should in the future liaise with vendors to adopt collaborative usability
mechanisms. Such mechanisms will encourage vendor-client collective understandings in dealing with
ineffective use of the system among strategic managers.
4. Contribution of the study
By addressing the research questions, one important contribution of the study was in identifying the
seven inhibitors we have earlier discussed, seen to have impacted negatively on the effective use of
ERP systems in the process industry. By ranking such inhibitors in the order of relevance, the study
presents non-collaborative training among employees as the top-most inhibitor to deserve manager’s
attention followed by other inhibitors as its second major contributions. By such rankings the
managers could prioritise their strategic interventions in line with the available resources earmarked
for IS continuous maintenance by the top management. These inhibitors are classified as internal
inhibitors; the only exception was poor vendors’ support that is classified as an external inhibitor.
Internal inhibitors can be resolved within the adopting organisation by the strategic managers since
their attentions have been drawn in this study using Delphi technique, while the external inhibitor need
to be addressed by collaborating with the vendors of ERP systems in this case SAP. These
contributions have addressed the extent to which the study has answered the research questions.
Besides, it has also addressed the limitations of the existing literature, which seems to concentrate
more on enablers of ERP usage adoption, with an implicit assumption that the opposite of such
enabling factors equates to inhibitors. By integrating the commonly used theories in IS, the study has
been able to categorize and rank the inhibitors that are qualitatively distinct from the enablers as
factors that could inhibit the effective use of ERP systems in New Zealand’s process industries.
5. Limitations
The study was not without limitations. Specifically, since the Delphi method requires participants with
varied expertise, the researcher had to rely on the recommendations of other experts considered as
motivators to populate the panels. This process is known as daisy chaining and has the potential to
form cliques. The study considers only process industries with SAP installed base; therefore the
findings might not be generalized to other ERP software such as Oracle or Microsoft ERP products
(Navision, Axapta), especially on findings related to ERP default attributes. Delphi multistage probing
requires considerable time, and, in addition, the fact that participants inevitably drop out after the first
round of the three round iteration process might slightly affect the result.
6. Conclusion and further research
The empirical findings of the study suggest that ERP adopters in New Zealand’s process industries
are not achieving the strategic business value identified in the project justification due to employees’
ineffective use of the systems. It has also further heightens the need for ERP adopters to evaluate
every phase of the ERP innovation process to understand the weakest link, especially in the
appropriation phase. Apparently, ERP adopters in New Zealand’s process industries have witnessed
implementation success, but this study takes it further to reveal that despite that, there are inhibitors
to the effective use of the systems.
By integrating TOE framework, TTF and IT inhibition theories as well as using the Delphi technique,
we highlight the top-most inhibitors that support ineffective use of the ERP systems in the
organisations seen as distinct and unique from the multiple arrays of enablers of ERP use that have
been studied in prior IS literature. With hardly any research in New Zealand to determine inhibitors of
effective use of ERP systems, this study has shown that while the firms have registered
implementation success, they are hindered from achieving the strategic business value because of
the prevailing constraints, some of which are more specific to New Zealand’s process industries,
especially in the areas of collaborative training among employees and poor vendors’ support.
Consequently, the next dimension that emerged in this study was the proffering of solutions to
overcome such prevailing inhibitors by the strategic managers seen as expert users who were eager
to go beyond the basic use of the ERP systems. By proffering such solutions for top management’s
attention by the managers themselves who occupy various strategic roles in the adopting
organisation, confirms the practical relevancy of the study, which has not been the focus of prior ERP
studies. Several IS researchers have highlighted the need for IS studies to be relevant to practitioners
as well as maintain the dictates of a rigorous academic research (e.g., Benbasat & Zmud, 1999;
Rosemann & Vessey, 2008). In summary, by way of answering the research questions, the study has
been able to: (1) identify the inhibitors plaguing New Zealand’s process industry, (2) prioritised them
for management’s attention, and (3) suggest practical solutions to overcome such inhibitors for
managerial interventions. Besides, for us to pay heed to the previous section remarks on limitations
of the study, a further research could consider other types of ERP software packages with reference
to other industries in New Zealand.
Acknowledgements
The author would like to thank the participants of 2011 IS Writer's Workshop organised by Information
Systems department of University of Cape Town (UCT) for reviewing this paper prior to submission.
References
Aarde, V. D. (2007) 'Enabling Supply chain transformation in the Pulp and Paper Industry', available:
https://cw.sdn.sap.com/servlet/JiveServlet/previewBody/6981-102-1-
12537/SPHR07EU_Sappi_APO.pdf;jsessionid [accessed December 24, 2008].
AberdeenGroup (2008) “ERP Options for related industries”, Aberdeen Research Group 2008.
Akkermans, H. A., Bogerd, P., Yucesan, E. and Wassenhove, L. N. (2003) “The impact of ERP on supply chain
management: Exploratory findings from a European Delphi study”', European Journal of Operational
Research, 146(284–301).
Al-Mashari, M., Al-Mudimigh, A. and Zairi, M. (2003) “Enterprise Resource Planning : A Taxonomy of Critical
Factors”, European Journal of Operational Research,, 146, 352-364.
Amoako-Gyampah, K. (2007) “Perceived usefulness, user involvement and behavioral intention: an empirical
study of ERP implementation”, Computers in Human Behavior, 23, 1232–1248.
Anderson, D. H. and Schneider, I. E. (1993) “Using the Delphi process to identify significant recreation research-
based innovations”, Journal of Park and Recreation Administration, 11(1), 25-36.
Benbasat, I., & Zmud, R. W. (1999) “Empirical research in information systems: The practice of relevance” MIS
Quarterly, 23(1), 3-16.
Bhattacherjee, A. and Hikmet, N. (2007) “Physicians' resistance toward healthcare information technology: a
theoretical model and empirical test”, European Journal of Information Systems, 16, 725–737.
Boudreau, M. C. (2002) “Learning to use ERP technology: a causal model”, the Proceedings of the 36 Hawaii
International Conference on System Sciences.
Bradley, L. and Stewart, K. (2002) “A Delphi study of the drivers and inhibitors of Internet banking”, International
Journal of Bank Marketing, 20(6), 250 - 260.
Broomfield, D. and Humphries, G. M. (2001) “Using the Delphi technique to identify the cancer education
requirements of general practitioners”, Medical Education, 35, 928-937.
Brown, S. A., Massey, A. P., MontoyaWeiss, M. M. and Burkman, J. R. (2002) “Do I really have to? User
acceptance of mandated technology”, European Journal of Information Systems, 11(4), 283-295.
Calisir, F. and Calisir, F. (2004) “The relation of interface usability characteristics, perceived usefulness, and
perceived ease of use to end-user satisfaction with enterprise resource planning(ERP)systems”,
Computers in Human Behavior, 20, 505–515.
Cantrill, J. A., Sibbald, B. and Buetow, S. (1996) “The Delphi and nominal group techniques in health services
research”, International Journal of Pharmacy Practice, 4(2), 67-74.
Cenfetelli, R. T. (2004) “Inhibitors and Enablers as Dual Factor Concepts in Technology Usage”, Journal of the
Association for Information Systems, 5(11-12), 472-492.
Cenfetelli, R. T. (2010) “Identifying and Testing the Inhibitors of Technology Usage Intentions”, Information
Systems Research, 1-19.
Chang, M.-K., Cheung, W., Cheng, C. and Yeung, J. H. Y. (2008) “Understanding ERP system adoption from the
user's perspective”, Int. J. Production Economics, 113, 928 – 942.
Chen, I. J. (2001) “Planning for ERP systems: Analysis and future trend”, Business Process Management
Journal, 7(5), 374-387.
Cohen, W. M. and Levinthal, D. L. (1990) “Absorptive capacity: A new perspective on learning and innovation”,
Administrative Science Quarterly, 35, 128-152.
Collins, J., Hanlon, A., More, S. J., Wall, P. G. and Duggan, V. (2009) “Policy Delphi with vignette methodology
as a tool to evaluate the perception of equine welfare”, The Veterinary Journal, 181, 63–69.
Davis, F. D., Bagozzi, R. P. and Warshaw, P. R. (1989) “User acceptance of computer technology: A comparison
of two theoretical models”, Management Science, 35(8), 982–1003.
DeLone, W. H. and McLean, E. R. (1992) “Information systems success: the quest for the dependable variable”,
Information Systems Research, 3(1), 60-95.
DeLone, W. H. and McLean, E. R. (2003) “The DeLone and McLean model of information systems success: a
ten-year update”, Journal of Management Information Systems, 19(4), 9-30.
Esteves, J. (2009) “A benefits realization road-map framework for ERP usage in small and medium-sized
enterprises”, Journal of Enterprise Information Management, 22(1/2), 25-35.
Esteves, J. and Bohorquez, V. (2007) “An Updated ERP Systems Annotated Bibliography: 2001-2005”,
Communications of AIS, 19(18).
Gable, G. G., Sedera, D. and Chan, T. (2003) “Enterprise systems success: a measurement model” in March, S.
T., Massey, A. and DeGross, J. I., eds., Proceedings International Conference on Information Systems,
Seattle, Washington: ICIS Press, 576-591.
Gummesson, E. (2000). Qualitative methods in management research. Thousand Oaks: Sage.
Gupta, A. (2000) “Enterprise Resource Planning: The emerging organizational value systems”, Industrial
Management & Data Systems, 100(3), 114–118.
Hakkinen, L. and Hilmola, O.-P. (2008) “Life after ERP implementation Long-term development of user
perceptions of system success in an after-sales environment”, Journal of Enterprise Information
Management, 21(3), 285-309.
Hameri, A. and Lehtonen, J. (2001) “Production and supply management strategies in Nordic paper mills”,
Scandinavian Journal of Management, 17, 379-396.
Hamerman, P. (2007) “ERP applications 2007: Innovation rekindles. Forrester Research”, available:
http://www.forrester. com/Research/Document/0,7211,41883,00.html [accessed December 18, 2008].
Hsieh, P.-A. J. J. and Wang, W. (2007) “Explaining employees' Extended Use of complex information systems”,
European Journal of Information Systems, 16, 216–227.
Hsu, C. and Sandford, B. A. (2007) “Minimizing Non-Response in The Delphi Process: How to Respond to Non-
Response”, Practical Assessment, Research & Evaluation, 12(17).
Hübner-Bloder, G. and Ammenwerth, E. (2009) “Key Performance Indicators to Benchmark Hospital Information
Systems – A Delphi Study”, Methods Inf Med, 48, 508–518.
Iansiti, M. (2007) 'ERP end-user business productivity: A field study of SAP & Microsoft. Keystone Strategy',
available: http:// download.microsoft.com/download/4/2/7/427edce8-351e-4e60-
83d628bbf2f80d0b/KeystoneERPAssessmentWhitepaper.pdf. [accessed December 18, 2008].
Ifinedo, P. (2008) “Impacts of business vision, top management support, and external expertise on ERP
success”, Business Process Management Journal, 14(4), 551-568.
Jasperson, J., Carter, P. E. and Zmud, R. W. (2005) “A Comprehensive Conceptualization of the Post-Adoptive
Behaviors Associated with IT-Enabled Work Systems”, MIS Quarterly, 29(3), 525-557.
Kamhawi, E. M. (2008) “Enterprise Resource-Planning Systems Adoption in Bahrain: Motives, Benefits, and
Barriers”, Journal of Enterprise Information Management., (21).
Kositanurit, B., Ngwenyama, O. and Osei-Bryson, K.-M. (2006) “An Exploration of Factors that Impact Individual
Performance in an ERP Environment: An Analysis using Regression Splines”, European Journal of
Information Systems, 15(3), 556-568.
Kouki, R., Poulin, D. and Pellerin, R. (2006) “ERP Assimilation Challenge: An Integrative Framework for a Better
Post-implementation Assimilation”, Working Paper DT-2006-DP-1: Interuniversity Research Center on
Enterprise Networks, logistics and transportation (CIRRELT).
Lail, P. (1999) “Idiosyncracies of papermaking create problematic ERP installations”, Pulp & paper, 73(6), 3.
Lassila, K. S. and Brancheau, J. C. (1999) “Adoption and Utilization of Commercial Software Packages: Exploring
Utilization Equilibria, Transitions, Triggers and Tracks”, Management Information Systems Quarterly,
16(2), 63-90.
Li, Y., Liao, X. W. and Lei, H. Z. (2006) “A Knowledge Management System for ERP Implementation”, Systems
Research and Behavioral Science, 23, 157-168.
Loftus, P. (2008) “Achieving High Performance Through Strategic Use of ERP”, North American Practice lead for
Accenture.
MacKinnon, W., Grant, G. and Cray , D. (2008) “Enterprise Information Systems and Strategic Flexibility”,
Proceedings of the 41st Hawaii International Conference on System Sciences.
Markus, M. L. and Tanis, C. (2000) “The Enterprise Systems Experience - From Adoption to Success” in Price, R.
W. Z. M. F., ed. Framing the Domains of IT Research: Glimpsing the Future Through the Past, Pinnaflex
Educational Resources, Inc.
Miaskiewicz, T. and Kozar , K. (2006) “The Use of the Delphi Method to Determine the Benefits of the Personas
Method – An Approach to Systems Design”', Proceedings of the Fifth Annual Workshop on HCI
Research in MIS, Milwaukee, WI.
Nah, F. F., Tan, X. and Teh, S. H. (2004) “An Empirical Investigation on End-Users' Acceptance of Enterprise
Systems”, Information Resources Management Journal, 17(3).
Nayan, N. M., Zaman, H. B. and Sembuk, T. M. T. (2010) “Defining Information System Failure in Malaysia:
Results from Delphi Technique”, IEEE computer society.
Nicolaou, A. I. (2004) “ERP Systems Implementation: Drivers of Post-Implementation Success”, in IFIP
TC8/WG8.3 International Conference Decision Support in an Uncertain and Complex World,
Okoli, C. and Pawlowski, C. (2004) “The Delphi method as a research tool: an example, design considerations
and applications”, Information & Management, 42, 15–29.
Pan, M. and Jang, W.-Y. (2008) “Determinants of the adoption of Enterprise resource planning within the
technology- organization- environment framework: Taiwan's communications industry”, Journal of
Computer Information Systems.
Ravichandran, T. (2005) “Organizational Assimilation of complex technologies: An empirical study of component
based software development”, IEEE Transactions on Engineering Management, 52, 249-268.
Robertson, M., Swan, J. and Newell, S. (1996) “The role of networks in the diffusion of technological innovation”
Journal of Management Studies, 3, 333–359.
Rogers, E. M. (2003) Diffusion of innovations, 5 ed., New York: Free press.
Rosemann, M., & Vessey, I. (2008). “Toward Improving the Relevance of Information Systems Research to
Practitioners: The Role of Applicability Checks” MIS Quarterly, 32(1), 1-22.
Schmidt, R. C. (1997) “Managing Delphi surveys using nonparametric statistical techniques”, Decision Sciences,
28(3), 763-74.
Schmidt, R. C., Lyytinen, K. J., Keil, M. and Cule, P. (2001) “Identifying software project risks: An international
Delphi study”, Journal of Management Information Systems, 17(4), 5 - 36.
Shah, N. (2005) “Process industry supply chains: Advances and challenges”, Computers and Chemical
Engineering, 29, 1225–1235.
Shahawai, S. S. and Idrus, R. (2010) “Pre-Considered Factors Affecting ERP System Adoption in Malaysian
SMEs using a Technology-Organization-Environment Framework”, IEEE computer society.
Smyth, R. W. (2001) “Challenges to successful ERP Use”, in The 9th European Conference on Information
Systems, Bled.,
Soja, P. and Paliwoda-Pekosz, G. (2009) “What are real problems in enterprise system adoption?” Industrial
Management & Data Systems, 109(5), 610-627.
Somers, T. and Nelson, K. G. (2003) “The impact of strategy and integration mechanisms on enterprise system
value: Empirical evidence from manufacturing firms”, European Journal of Operational Research, 146,
315–338.
Sousa, R. D. and Goodhue, D. L. (2003) “Understanding exploratory use Of ERP systems”, Ninth Americas
Conference on Information Systems.
Strauss, A. and Corbin, J. (1998) Basics of qualitative research, Thousands Oaks, CA: Sage Publications.
Thompson, O. (2000) Supply Chain Planning— Issues for Continuous Chemical Companies.
Topi, H., Babaian, T. and Lucas, W. (2005) “Identifying usability issues with an ERP implementation”, In
Proceedings of the Seventh International Conference on Enterprise Information Systems, Miami, FL.
Topi, H., Babaian, T. and Lucas, W. (2006) “Using informal notes for sharing corporate technology know-how”,
European Journal of Information Systems, 15(5), 486-499.
Tornatzky, L. and Fleischer, M. (1990) The process of technology innovation, Lexington, M.A: Lexington Books.
Van Everdingen, Y. (2000) “ERP adoption by European midsize companies”, Communications of the ACM, 43(4),
27-31.
von der Gracht, H. A., Jahns, C. and Darkow, I.-L. (2008) The Future of Logistics - Scenarios for 2025, Gabler
edition Wissenschaft ed., Gabler Verlag.
Wang, R., Hamerman, P., Thomas, Z. and Donnelly, M. (2008) “Topic Overview: ERP Applications 2008”,
Forrester Research.
Yeoh, W., Koronios, A. and Gao, J. (2008) “Managing the Implementation of business Intelligence Systems: a
Critical Success factors framework”, International Journal of Enterprise Information Systems, 4(3).
Yu, C.-S. (2005) “Causes influencing the effectiveness of the post-implementation ERP system”, Industrial
Management & Data Systems, 105(1), 115-132.
Zhu, Y., Li, Y., Wang, W. and Chen, J. (2010) “What leads to post-implementation success of ERP? An empirical
study of the Chinese retail industry”, International Journal of Information Management, 30, 265–276.
I thank you for agreeing to take part in this Delphi 3 round study. We are starting with the
brainstorming session that will take an hour of your time. Having made huge investments in
implementing ERP systems, the study aims to identify the supposed barriers that could restrain the
users from maximizing the business value of the systems in New Zealand’s process industries.
The questions for our deliberation are: “How can we explore the use of SAP beyond the basic use we
are currently doing? Have we maximized SAP capabilities in our everyday job challenges if not why?.
I posted these questions to you all prior to the brainstorming session for your reflections. To avoid
cliques, please do not share your reflections with any of your colleagues prior to the brainstorming
session.
The venue for the brainstorming session is Waikato Management School auditorium; the attached
camus map will assist you to easily locate it. It will begin at 10:30 am on May 15 2008
Once we have received responses from all participants during the brainstorming session your tape
recorded voice will be transcribed and coded to derive themes and formulate the first questionnaire,
which you should receive within the next two weeks. To minimise the time between successive rounds
of the Delphi survey it would be appreciated if you could return each questionnaire within two weeks
of receipt via email: gerrychi@gmail.com
We would like to remind you that individual responses will be strictly confidential to the research team
and will not be divulged to any outside party.
If at any time you would like to discuss the Delphi study, please do not hesitate to contact me.
Regards
Chidi Ononiwu
Delphi Survey to Identify the Barriers to Effective Use of Process Industries’ ERP Systems in
New Zealand
Thank you very much for your 100% attendance in the first round of our research and the tremendous
contributions you made during the brainstorming session. Based on that, we have identified nine key
barriers to effective use of process industry ERP in New Zealand, namely:
1) Inadequate ERP expertise
2) ERP default attributes
3) Non-collaborative training among employees
4) Low absorptive capacity
5) Poor user involvement
6) Lack of continuous improvement
7) System misfit
8) Consultant ineffectiveness
9) Poor vendor support
I converted these nine key barriers into questionaries as shown below for the second round of the
Delphi study. Please indicate your answers by means of a tick in the appropriate box. If there are any
objections to these barriers we have gleaned from the brainstorming session you are highly
encouraged to comment on them.
All information provided will be treated in the strictest of confidence and in line with ethic codes of the
university. If you do not want to receive further emails on this subject you are equally free to indicate.
Regards
Chidi Ononiwu
Please check which ERP system you use SAP Oracle Elixir Microsoft Best of
in your organisation. You can tick more Navision / breed (BoB)
than one depending on your usage. Axapta and
Others
Delphi Survey to Identify the Barriers to Effective Use of Process Industries’ ERP Systems in
New Zealand
Thank you very much for your reply in the second round of our research. This is the final round of the
Delphi survey. Generally, it has been very impressive and I commend you all for that. However, we
have not yet reached consensus. In the previous rounds we only had two drop outs citing lack of time
to continue. Please see the results of round 2 and if there are any objections you are free to
comment.
Based on the result of roll 2 we now have 7 key barriers. You are now asked to rank these 7 key
barriers in the order of importance for managers’ attention. The ranking round will be done three times
until we achieve consensus. Also you have to comment on how the barriers reflect your present
company situation and suggest ways of improving them. We encourage you to do this because as
Gummesson (2000) put forth “… those who are closest to a problem are also those who are best
suited to identify the problem and suggest solutions” (p.41).
Regards
Chidi Ononiwu
Keywords: broadband, South Africa, broadband benefits, broadband usage, broadband impact
1. Introduction
South Africa is considered one of the world’s leading emerging economies and is part of the Brazil,
Russia, India, China and South Africa (BRICS) group of emerging economies (BRICS, 2012).The
country has vast opportunities within the telecommunications sector and has seen much technological
advancement, yet it has been undermined by low internet penetration rates below 14%
(InternetWorldStats, 2012). The South African government has stated that the Information and
Communication Technologies (ICT) industry is fundamental to the country’s development; however,
regulations and policies have not always supported this stance (Francis, 2010). Moreover, once
regarded as a telecommunication powerhouse, South Africa is slowly losing ground to its African
counterparts such as Botswana, Zambia and Kenya due to stagnant inland infrastructure
developments (Balancing-Act, 2010). While the demand for broadband is high, the adoption of the
technology is lower than anticipated and this particularly within the households of consumers (Muller,
2010). Figures for broadband adoption in 2011 were a mere 3.6%; with fixed wired connections 1.6%;
multiple times lower than the OECD countries average of 25% (Department of Communications,
2011; Muller, 2011b; OECD, 2012).
Various studies and models have aimed to explain the adoption of broadband services in several
countries. Large-scale qualitative studies of the barriers to broadband adoption in the United States
analysed broadband adoption among low-income communities (Dailey et al., 2010; Powell, Bryne and
Dailey, 2010) and recommended a renewed focus on factors that sustain home broadband access
and usage (Powell, Bryne and Dailey, 2010). Broadband access is seen as a requirement of socio-
economic inclusion and that while usage is impacted by high monthly fees, there are also concerns
around hardware costs, hidden fees, billing transparency, quality of service, and availability (Dailey et
al., 2010). An economic study in the United States recommended targeted programs that educate
households about broadband benefits (Rosston, Savage and Waldman, 2010). In their study they
noted that experienced broadband users are more aware of Broadband benefits.
There has been little research focus on broadband adoption within the African continent and even
less on the use and impact of broadband services in African countries after the adoption process
occurs. To address this need, this research investigates the use and impact of broadband services in
South Africa. The research started with a literature review, followed by a qualitative study which was
ISSN 1566-6379 134 ©Academic Publishing International Ltd
Reference this paper as: Lisa F Seymour and Mogen Naidoo “The Usage and Impact of Broadband: A South
African Household analysis” The Electronic Journal Information Systems Evaluation Volume 16 Issue 2 2013,
(134-147) , available online at www.ejise.com
Lisa F Seymour and Mogen Naidoo
subsequently validated by a quantitative study. Through a review of the literature various models to
understand household broadband use and impact were identified and reviewed so as to propose a
model for broadband use and impact.
2. Literature review
Broadband is widely used as shorthand for high-speed Internet access (Kim, Kelly and Raja, 2010;
Sacks, 2002). Broadband can also be defined in terms of the technology used, in terms of the
transmission capacity provided, or in terms of the functionality enabled (Kim, Kelly and Raja, 2010;
Sacks, 2002). Compared to traditional narrowband connections, broadband access is immediate.
Large volumes of data can be instantly transmitted, waiting times are reduced and efficiency for users
is improved. In contrast narrowband connections are slow where it is estimated that one third of user
time online is spent waiting (Spurge and Roberts, 2005). For the purpose of this research broadband
is regarded as “an always available, multimedia capable connection with a download speed of at least
256 kbps” as defined by the South African Department of Communications (Department of
Communications, 2010:7). Broadband technologies are divided into fixed line technologies, fixed
wireless technologies and wireless mobile technologies (Corning, 2005). Fixed line technologies
include digital subscriber lines (DSL), cable modems, broadband over power lines (BPL) and fibre
optics such as fibre to the home (FTTH). Fixed wireless technologies include satellite, wireless fidelity
(Wi-Fi) and worldwide interoperability for microwave access (WiMAX). Wireless mobile technologies
include 3G and its evolutionary paths 3.5G, high speed data packet access (HSDPA), high speed
packet access plus (HSPA+) and 4G or long term evolution (LTE). ADSL is the most popular of the
fixed line technologies while Wi-Fi and 3G/HSDPA are the most popular of the fixed wireless and
mobile wireless technologies respectively (Corning, 2005).
This poor performance in broadband uptake needs to be looked at in contrast with repeated promises
from Government to deliver on supporting broadband infrastructure. Recently, the South African
Communications minister, Dina Pule, in her 2012 budget speech, stated that “the Department of
Communications remains committed to delivering 100% broadband penetration by 2020 and
delivering a million jobs by 2020” (Muller, 2012). However similar promises have been made by the
Government since 2006 and analysts have criticised the government for lack of delivery and too much
Yet, Shih and Venkatesh (2004) state that previous diffusion studies have focused mostly on the
adoption perspective. There are limitations to the adoption diffusion (AD) of technology and though
the diffusion processes cannot be understood without studying the nature of adoption, to complete the
diffusion story, use-diffusion processes also need to be examined (Shih and Venkatesh, 2004). Shih
and Venkatesh’s (2004) research extends the diffusion concept further with a systematic study of
post-adoption usage. In their model, the variable of interest is use or, more specifically, rate of use
and variety of use. The theoretical elements for their model are the evolving nature of use (rate and
variety), sustained continuous use and technology outcomes (technology integration, perceived
essentialness of technology, impact of technology, and user proneness to adopt new technologies)
(Shih and Venkatesh, 2004). Choudrie and Dwivedi, (2004a) argued that researchers have
demonstrated the impact of various technologies such as the Internet on a user’s daily life and
considering that broadband offers an alternative way of work and entertainment, it is likely to impact a
user’s daily activities. According to Choudrie and Dwivedi (2004b), broadband users are different from
narrowband users in terms of the time spent on the Internet and the activities conducted using the
Internet.
2.4 A proposed model for broadband use and impact in South African households
Figure 1: Proposed model for broadband use and impact in the household
Using the Model of Adoption of Technology in the Home (Venkatesh and Brown, 2001), removing the
adoption components and combining literature outcomes, the proposed model was formulated (Figure
1). A feedback loop was included, as through the continued use of broadband the outcomes of
broadband usage has a continued changing effect on the rate and variety of use of broadband by the
user. In the case of rate and variety of use, the effect can cause a user to move within the topology.
An example would be where a limited user can move to become a specialised or an intense user. It
therefore becomes necessary to investigate how the impact of broadband affects the rate and variety
of use through the feedback loop.
3. Research questions and approach
This study attempts to answer the following questions:
How do South African consumers use broadband?
What impact does broadband have on South African consumers and their interest in future
oriented technologies?
The overall research study makes use of multiple philosophies i.e. an interpretative and a positivist
philosophy. Each philosophy when used independently in research has its advantages as well as its
disadvantages. Therefore in order to attain accurate findings in any study it becomes necessary to rid
the study of the disadvantages inherent in a particular philosophy (Lee, 1991). Therefore taking Lee’s
(1991) framework into consideration, this study first uses an interpretative qualitative approach
followed by a positivist quantitative approach and then integrates them to ensure that the benefits of
each study are realised. The qualitative study attempted to expand and enhance through semi-
structured interviews the proposed model provided by the initial literature review. The data analysis
compiled the data collected into patterns and themes and structured the data into a framework
(Attride-Stirling, 2001). Using the input from the qualitative analysis, hypotheses were developed. The
next step was to validate the model through the use of the developed questionnaire. Participants in
South Africa were sent questionnaires and the data obtained was analysed quantitatively using
statistical techniques. Relationships between the different variables within the model were tested to
develop a model of broadband use and impact in households of South Africa. While the qualitative
study preceded the quantitative study the results will be discussed together in this paper.
4. The qualitative study
To select interviewees, a purposive sampling method was used with selection criteria developed with
the research questions in mind. Interviews were conducted with 16 respondents, six adopters who
were also users (C1-6), one user (U1) and nine non-adopters (N1-9) (Table 1). All respondents had
formal education ranging from university degrees to college diplomas. Most of these formal
educations made way for their occupational roles. Occupational roles within the IT industry ranged
from being lecturers of IT, systems architects and software developers to students in IT. The
respondents were therefore given the title of either IT or non-IT depending on their occupational
classification. The respondents who were in the high and medium income brackets had professional
occupations while those in the lower income bracket were university students or casual workers in
retail.
The study followed an inductive approach to analyse the data collected resulting in categories
(Thomas, 2006). Constant comparative analysis (Anfara, Brown and Mangione, 2002) was performed
to ensure coherence. To ensure that the data collected was reliable and valid, a triangulation of
interviews was performed with audio tapes and notes taken. The qualitative study also provided a
general idea of whether the issues are comparable to those experienced in other countries and
provided an indication of the expected relationship between variables. Furthermore, it was expected
that some factors would correlate with those identified in the literature survey and some additional
new factors/constructs would be discovered.
A pilot questionnaire was devised to resolve any difficulties that respondents faced when completing
the questionnaire and was then initially sent to 20 respondents who have experience in research
studies. For the final sample, a stratified random sample was used to look at distinct sub groups
based on age, occupation and income of adopters and non-adopters. The demographics of the
respondents are shown in Table 2. Of the 177 respondents, 155 had Internet access at home and 22
did not. Only one respondent had a dialup or narrowband connection while the rest had a broadband
connection. It should be noted that of the 154 respondents that did have a broadband connection, 32
respondents had more than one type of broadband connection in their household.
In terms of satisfaction, respondents felt that broadband allowed them to have more control over their
lives, they were less frustrated than when they used the slower dial-up connections as it makes life
easier and gets things done.
“It’s just great to have that, but come to think of it perhaps it is. It’s not frustrating. If
you want to know something you can search for it” (U4).
Five respondents showed interest in future technologies. This was mostly interest in faster
connections as well as interest in wireless connections as respondents C3 and C5 mentions
respectively:
"Well, I am a techno junkie. I would like to see what the future holds in terms of speed
in terms of how fast the connections going to be in the future. If I had money right now
I would get myself a 4Mb connection to see what is going on" (C3).
"I wouldn’t mind trying 3G. I would like to enquire more about it but about getting more
knowledge about it. I think it needs to be more widely exposed here. It has taken off
overseas but has taken us quite a while to adopt it here" (C5).
One respondent, however, explained that she is satisfied with her current broadband service and that
she currently has no interest in future technology:
"At the moment I don’t need faster or other technologies as its suits me fine" (C1).
From the interviews interest in the following six future technologies was identified:
higher speed lines
unlimited bandwidth
IPTV
High speed wireless
VOIP
Interactive online communication
8. Quantitative findings on impact of Broadband
The six hypotheses derived from the literature and the qualitative finds were:
H1: The greater a user’s rate and variety of use, the easier it is for users to communicate with
distant families. [Lifestyle: Communication]
H2: The greater a user’s rate and variety of use, the more a user will be able to save time and
hence commence with other life activities. [Lifestyle: Life Activities]
H3: The greater a user’s rate and variety of use, the more a user will be able to save on costs.
[Lifestyle: Costs]
H4: The greater a user’s rate and variety of use, the easier a user will be able to perform work
related activities at home. [Lifestyle: Work Activities]
H5: The greater a user’s rate and variety of use, the more satisfied a user will be with the
technology. [Satisfaction]
H6: The greater a user’s rate and variety of use, the higher the level of interest in acquiring
futuristic-oriented technologies. [Future technologies]
Table 3 and Table 4 present the means and standard deviations of aggregate measures for the
quantitative analysis. Respondents seemed to have a medium agreement (scores between 5.5 and
6.5) with the satisfaction factor and two lifestyle factors, namely work activities and communication
(Table 3). Respondents showed weak agreement for the remaining two lifestyle factors, cost and life
activities. Respondents showed interest (scores greater than 3.5) in future technologies such as
higher speed lines, unlimited bandwidth with no shaping, paradigm shift from traditional to IPTV and
high speed wireless connections(Table 4). Respondents were somewhat interested (2.5 to 3.5) in
VoIP and interactive online communication systems.
Table 3: Summary of descriptive statistics for lifestyle and satisfaction factors (items were scaled from
1 being strongly disagree to 7 being strongly agree)
Rank Construct/Items Items Respondents Mean Standard
deviation
1 Work Activities 1 154 5.84 1.4
2 Communication 1 154 5.75 1.5
3 Satisfaction 2 154 5.67 1.5
4 Costs 1 154 5.30 1.7
5 Life Activities 1 154 4.89 1.8
Table 4: Summary of descriptive statistics for interest in future technologies (items were scaled from 1
being not at all to 4 being very interested)
Rank Construct/Items Items Respondents Mean Standard
deviation
1 Higher speed lines 1 154 3.97 0.2
2 Unlimited bandwidth no shaping 1 154 3.95 0.3
3 Paradigm shift from traditional TV to 1 154 3.57
0.8
IPTV
4 High speed wireless 1 154 3.56 0.8
5 Voice over IP 1 154 3.37 0.9
6 Interactive online communication 1 154 3.23
1.0
systems
Four (H2, H4, H5 and H6) of the six hypotheses were partially supported by the data that explained
broadband's impact on consumer’s lives (Figure 6). The hypotheses that were partially supported
were due to the data either only supporting a consumer’s rate of use or variety of use. The supported
results are listed below.
Since variety of use has a significant relationship with consumers saving time and hence being
able to commence with other life activities, the lifestyle impact hypothesis H2 is partially accepted.
The lifestyle impact hypothesis H4 is partially accepted due to users having a high rate of use
performing work related activities at home using their broadband subscription.
Since there is a significant relationship between a consumer’s rate of use and satisfaction with
their broadband subscription, the satisfaction hypothesis H5 is partially accepted.
Since there is a significant relationship between a consumer’s rate of use and interest in
interactive online communication, higher speed wireless technologies, IPTV, VOIP and unlimited
bandwidth, the hypothesis H6 is partially accepted.
The results show that broadband usage impacted lifestyle, satisfaction and interest in future
technologies. South African consumers perceive that they have a more comfortable lifestyle when
using broadband, are more satisfied and show an interest in future-oriented communication
technologies.
in order to examine the usage behaviour of consumers after they had adopted the technology. Due to
time constraints a cross-sectional study was performed. It is therefore recommended that for future
research a longitudinal study be performed. As a result, limitations to fully study the impact of
broadband would be overcome. In particular the feedback loop which was out of scope of this
research study. In addition, longitudinal research will also help in identifying in greater detail the actual
usage habits of broadband users over time.
From a practical viewpoint this research provided an analysis into the usage of broadband in
residential households that may be beneficial for various stakeholders such as government, Internet
Service Providers (ISP), business consumers and public organisations. The categorising of adopters
into the four use-diffusion categories provides an effective way to segment markets for high-tech
products. South African consumers were still found to be experimental users.
Over the years the impact of broadband has become a very broad area of research, while much
research has been done to look at the economic effects of broadband usage, this study looks at the
impact on individuals. Broadband use was shown to enable working from home and resulted in
increases in time saving; confirming previous studies on digital citizenship. However future research
to assess whether consumers are starting to access online education and online government services
would be valuable. There is also a need to research specific areas such as new communication
applications and downloads and entertainment, in order to determine the real impact of broadband.
Moreover, other specific areas such as the diffusion and sustainability of broadband technology,
family and work life, social networks and online security and privacy will need to be explored.
Due to a high mobile phone penetration rate in South Africa, there are large number of people who
are accessing mobile broadband data services in the rural areas of South Africa. These users may not
have the same income and education levels as the sample of this study. As such, future research may
be deemed important to investigate the use of mobile broadband data services in more rural parts of
South Africa.
References
Akamai. (2010) The State of Internet, 2nd Quarter, 2010 Report, vol. 2, no. 2. [Online], Available:
http://www.akamai.com/stateoftheinternet/ [06/01/2012]
Anfara, V. A., Brown, K. M., & Mangione, T. L. (2002). ‘Qualitative analysis on stage: Making the research
process more public’. Educational Researcher, vol. 31, no.7, pp. 28-38.
Attride-Stirling, J. (2001) Thematic networks: an analytic tool for qualitative research. Qualitative Research, vol. 1,
no. 3, pp. 385-405. SAGE Publications.
Ajzen, I. (1991) ‘The theory of planned behaviour’, Organizational Behavior and Human Decision Processes, no.
50, pp. 179–211.
Ajzen I. and Fishbein, M. (1980) ‘Understanding Attitudes and Predicting Social Behaviour’, Englewood Cliffs, NJ:
Prentice Hall.
Balancing-Act. (2010) ‘SA May Lose Out on Wireless Broadband Boom’, Issue 260, [Online], Available:
http://www.balancingact-africa.com/news/en/issue-no-260/internet/sa-may-lose-out-on-
w/en, [29/01/2011].
BRICS (2012) BRICS Information Centre. [Online], Available: http://www.brics.utoronto.ca/ [30/06/2006]
Bernabé, F. (2010) ‘Broadband Quality Score III – A global study of broadband quality’, Oxford SAID Business
School, CISCO, September 2010.
Choudrie, J. and Dwivedi, Y. K. (2004a) ‘Towards a Conceptual Model of Broadband Diffusion’, Journal of
Computing and Information Technology – CIT 12, 2004, 4, pp. 323-338.
Choudrie, J. and Dwivedi, Y. K. (2004b) ‘Analyzing the Factors of Broadband Adoption in the Household’,
Research Paper, Brunel University, [Online], Available:
http://csrc.lse.ac.uk/asp/aspecis/20040034.pdf [30/06/2006].
Corning. (2005) ‘Broadband Technology Overview’, White Paper, WP6321, [Online], Available:
http://www.corning.com/docs/opticalfiber/wp6321.pdf [30/06/2006].
Dailey, D., Bryne, A., et al. (2010) Broadband adoption in low-income communities. Social Science Research
Council, Brooklyn, NY. [Online], Available:
http://webarchive.ssrc.org/pdfs/Broadband_Adoption_v1.1.pdf. [06/01/2012].
Davis, F. D. (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology,
MIS Quarterly, no. 13, pp. 319–340.
Department of Communications (2010) Broadband Policy for South Africa [Online], Available:
http://www.info.gov.za/view/DownloadFileAction?id=127922 [01/06/2012].
Department of Communications (2011) e-Barometer report. [Online], Available: http://www.doc.gov.za
[01/06/2012].
Dwivedi, Y. K. (2008) ‘Consumer Adoption and Usage of Broadband’, Idea Group Inc (IGI), ISBN 1599047853,
9781599047850.
Dwivedi, Y.K., Khoumbati, K., Williams, M.D., & Lal, B. (2007). Factors affecting consumers’ behavioural intention
to adopt broadband in Pakistan. Transforming Government People, Process and Policy, vol. 1, no.3, pp.
285-297.
Dwivedi, Y. K., Williams, M. D., Lal, B., Weerakkody, V., & Bhatt, S. (2008). Understanding factors affecting
consumer adoption of broadband in India: A pilot study. In proceeding of the 5th International Conference
on E-Governance, Hyderabad, India, 28th–30th December 2007.
Francis, L. (2010) ‘Broadband for all not likely’, ITWeb Financial, [Online], Available:
http://www.itweb.co.za/index.php?option=com_content&view=article&id=34965%3Abroa
dband-for-all-not-likely&Itemid=132 [20/12/2010].
Goldstuck, A. (2010) ‘Internet access in South Africa 2010’, Johannesburg: World Wide Worx. [Online] Available:
www.worldwideworx.com.
InternetWorldStats. (2012) ‘Internet usage statistics for Africa’, [Online], Available:
http://www.internetworldstats.com/stats1.htm#africa [30/05/2012].
Kim, Y., Kelly, T., and Raja, S. (2010) ‘Building broadband: Strategies and policies for the developing world’,
Global Information and Communication Technologies Department, World Bank, 1818H Street NW, MSN
F5K 508, Washington DC 20433, USA.
Lee, A.S. (1991) ‘Integrating Positivist and Interpretive Approaches to Organizational Research’ Organization
Science, Issue 2, pp. 342-365.
Middleton, C.A. and Chang, S. 2008. The Adoption of Broadband Internet in Australia and Canada. In Handbook
of Research on Global Diffusion of Broadband Data Transmission, edited by Dwivedi, Y. K.;
Papazafeiropoulou, A. and Choudrie, J. Harrisburg, PA: IGI Global. 820-842.
Middleton, C. 2009. ‘Can broadband support environmental sustainability? An exploration of claims at the
household level’. Telecommunications Journal of Australia. Vol. 59, no. 1, pp. 10.1 to 10.14. DOI:
10.2104/tja09010.
Mossberger, K. Tolbert, C. and McNeal R. (2008) Digital Citizenship: The Internet, Society, and Participation. MIT
Press
Muller, R. (2010) ‘SA’s broadband penetration: The way forward’, [Online], Available:
http://mybroadband.co.za/news/broadband/15804-SAs-broadband-penetration-The-
way-forward.html, [01/06/2012].
Muller, R. (2011a) ‘Broadband Speeds: SA versus the World’, [Online], Available:
http://mybroadband.co.za/news/broadband/37252-broadband-speeds-sa-versus-the-
world-2.html [01/06/2012].
Muller, R. (2011b) ‘Broadband penetration and GDP: SA still sucks’, [Online], Available:
http://mybroadband.co.za/news/broadband/39777-broadband-penetration-and-gdp-sa-still-sucks.html
[01/06/2012].
Muller, R. (2012) ‘The broadband pipe dream’, [Online], Available:
http://mybroadband.co.za/news/broadband/50115-the-broadband-pipe-dream.html [01/06/2012].
OECD (2012). OECD Broadband Portal. [Online], Available:
http://www.oecd.org/document/54/0,3746,en_2649_37441_38690102_1_1_1_37441,00.html [06/01/2012]
Ooi, K.-B., Sim, J.-J., Yew, K.-T. & Lin, B. (2011) ‘Exploring factors influencing consumers’ behavioral intention to
adopt broadband in Malaysia’. Computers in Human Behavior (2011), vol. 27, no. 3, pp. 1168-1178.
Powell, A., Bryne, A. and Dailey, D. (2010) ‘The Essential Internet: Digital Exclusion in Low-Income American
Communities,’Policy & Internet: Vol. 2: Iss. 2, Article 7. [Online], Available:
http://www.psocommons.org/policyandinternet/vol2/iss2/art7 [06/01/2012]
Reed, K. (2011). VoIP flourishing despite bandwidth limitations. [Online], Available:
http://www.techleader.co.za/karlreed/2011/07/06/voip-flourishing-despite-bandwidth-limitations/
Rogers, E. (1995) Diffusion of Innovations, 4th Edition. USA: Free Press.
Rosston, G. L.; Savage, S. J.; and Waldman, D. M. (2010) ‘Household Demand for Broadband Internet in
2010,’The B.E. Journal of Economic Analysis & Policy: Vol. 10: no. 1 (Advances), Article 79. [Online],
Available: http://www.bepress.com/bejeap/vol10/iss1/art79 [06/01/2012]
Sacks, G. (2002) ‘Determinants of Broadband Adoption in the Health Care Industry’, Dissertation Paper,
Department of Information Systems, University of Melbourne, Australia, [Online], Available:
http://www.dis.unimelb.edu.au/postgraduate/theses/2002/SACKS(2002)_abstract.pdf
[01/06/2006].
Shibeshi, Z.S., Terzoli, A. and Bradshaw, K. (2011) ‘Towards a Toolkit for creating video-oriented services for
Mobicents’, Proceedings of the Southern Africa Telecommunication Networks and Applications Conference
(SATNAC) 2011, [Online], Available:
http://www.satnac.org.za/proceedings/2011/papers/Network_Services/200.pdf
Shih, C. F. and Venkatesh, A. (2004) ‘Beyond adoption: Development and application of a use-diffusion model’,
Journal of Marketing, no. 68, pp. 59–72.
Straus, J. in coll. with Holzapfel, H. and. Lindenmeir, M. (2004) ‘Genetic Inventions and Patent Law: An Empirical
Survey of Selected German R & D Institutions’, VMD Verlag Medien Design, Munich.
Spurge, V. and Roberts, C. (2005) ‘Broadband Technology: An Appraisal Of The Availability And Use By Small
And Medium Sized Enterprises’, Department of Real Estate & Construction, Oxford Brookes University,
Oxford, [Online], Available:
http://www.prres.net/papers/Spurge_Broadband_Technology_An_Appraisal_Of_The_Av
ailability.Pdf [04/12/2009].
Taylor, S. and Todd, P. A. (1995) ‘Understanding Information Technology Usage: A Test of Competing Models’,
Information Systems Research, no. 6, pp.44–176.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal
of Evaluation, vol. 27, no. 2, 237.
Venkatesh, V.& Brown, S. (2001) ‘A longitudinal investigation of personal computers in homes: adoption
determinants and emerging challenges’, MIS Quarterly, no. 25, pp. 71–102.
Vorster, G. (2012) ‘Study critical of SA govt ICT spend’. [Online], Available:
http://businesstech.co.za/news/government/14067/study-critical-of-sa-govt-ict-spend/
Wieck, R. and Vidal, M. (2011) ‘Investment in telecommunications infrastructure, growth and employment –
recent research’, International Journal of Management and Network Economics, vol. 2, no. 2, pp. 135-149.
Abstract: The critical importance of Enterprise Resource Planning (ERP) systems in modern
business has created a demand for ERP consultants with the appropriate competencies to implement,
maintain and support these systems. Education and training programmes have been implemented in
order to provide ERP specialists and graduates with the required industry relevant ERP
competencies. The majority of these education and training programmes utilise large ERP systems for
instructional purposes, however users of these systems encounter usability issues whilst learning to
use the systems. The use of medium-sized ERP systems has been proposed for educational
purposes as they are less complex and easier to learn than large ERP systems. Empirical studies on
the usability of ERP systems, particularly for medium-sized ERP systems are limited. This paper
reports on empirical research on the usability evaluation of a medium-sized ERP system. The study
identified three categories of criteria and 10 criteria which can be used for usability evaluations of
medium-sized ERP systems. The criteria were used in a case study to evaluate the usability of a
medium-sized ERP system and to obtain qualitative feedback on the usability of the system. The most
frequently reported positive usability features of the ERP system were the tree-structure of the menus
and the grouping of logically related items. Negative features which were reported included the clutter
of the user interface and difficulties with finding information and controls. These results can provide
valuable insight into the ERP learning process for university educators and researchers. The usability
evaluation results can assist ERP designers with improving ERP usability, which can improve the
quality of ERP training and education programmes and ultimately ERP project success. The usability
evaluation results provide considerable insight into the usability problems encountered by students
when learning to use ERP systems in their university courses and provide a valuable contribution to
usability theory and in particular frustration theory.
Keywords: ERP usability, learning ERP; navigation of ERP systems; ERP education; ERP system
evaluation
1. Introduction
An Enterprise Resource Planning (ERP) system is defined as ‘a highly unified, consolidated and
reliable network of business systems, built on a single integrated platform’ (Vaman, 2007). The
removal of barriers to sharing information between functional divisions and the holistic management
of processes have enabled ERP systems to increase the operational processes, profitability and
productivity of organisations (Magal and Word, 2012; McGauhey and Gunasekaran, 2007). In spite of
the economic recession over recent years, ERP systems are dominant in the marketplace and this
trend is expected to continue (Forrester, 2011). Small to medium enterprises (SME) have
implemented ERP systems and vendors have introduced less complex, medium-sized ERP systems
such as Sage, SYSPRO and Softline Accpac, to suit the requirements of these organisations. Large
ERP systems are often too sophisticated and complex for smaller organisations, although vendors
such as SAP and Microsoft have introduced new or scaled down versions aimed specifically at the
SME market (Winkelmann and Matzner, 2010).
ERP implementation success can be affected by the level of appropriate specialised competencies
within the ERP project team (Ifinedo and Nahar, 2009; Rothenberger et al., 2010). Organisations that
recruit ERP specialists require graduates who have industry-relevant ERP competencies. The ERP
competencies related to the various ERP systems must be acquired by ERP specialists and
graduates of undergraduate Information Systems (IS) degrees, in order to implement and support
business applications (Boyle and Peslak, 2010; Scholtz et al., 2011). An increased demand for ERP
specialists with the appropriate competencies exists internationally (Beer, 2010; Law et al., 2010;
Ngai et al., 2008) and in South Africa (Scholtz et al., 2011; Seymour et al., 2006).
In response to the demand for ERP specialists, several ERP training and education programmes
have been developed which incorporate ERP systems for instructional purposes (Hustad and Olsen,
2011; Seethamraju, 2007; Surendran and Somarajan, 2006). However, the practical use of the ERP
systems in these educational programmes is often accompanied with students experiencing learning
difficulties when learning to use these complex ERP systems (Seethamraju, 2007; Theling and Loos,
2005) which are not designed to support teaching (Shtub, 2001). Surendran and Somarajan (2006)
reported that the adoption of an ERP system in the curricula in higher education improved student
competencies but the ease of use of the ERP system was rated poorly by the students. Another study
by Seethamraju (2007) showed that students had an improvement in transactional competencies but
struggled to understand the underlying business process flow of the ERP system tasks due to the
complexity of the user interface. Studies into the usability of ERP systems used by students in higher
education can improve the usability of these tools.
Several studies in industry have revealed that the complexity of ERP systems has resulted in user
interfaces (UIs) which suffer from poor usability (Singh and Wesson, 2009; Yeh, 2006) and user
frustration (Matthews, 2008; Topi et al., 2005). The poor usability makes it difficult for users to interact
with the ERP system and to complete required tasks, which further impacts the time taken to learn the
system (Topi et al., 2005). Usability problems with ERP systems, similar to those reported in industry,
have also been encountered by training and education institutions (Surendran and Somarajan, 2006).
The learning curve for students is usually steep as students have to master the many menu paths
within a single ERP system (Theling and Loos, 2005). Studies of ERP system adoptions in higher
education programmes have reported that medium-sized ERP systems are less complex and easier
to use than large ERP systems (Hustad and Olsen, 2011; Winkelman and Matzner, 2010). However
these studies have been largely qualitative.
The potential benefits of ERP systems for organisations can be increased by improving the usability of
ERP systems which can, in turn, reduce the length of the training time and increase employee user
satisfaction (Topi et al., 2005). Continued research into the improvement of ERP system’s usability is
required in order to provide user interfaces for ERP systems which are easy to use, easy to learn and
support the user’s tasks. In particular field studies which focus on users’ perceptions of ERP systems
are required which can improve the understanding of the factors that affect usability. Research into
improving the usability of ERP systems in education is required in order to address the complexity and
learning curve problems of ERP systems.
Usability evaluation is any analysis or empirical study of the usability of a prototype or system (Foltz et
al., 2008). Several studies (Costa, 2010; Singh and Wesson, 2008; Van Norren, 2009) have proposed
usability criteria which can be used for usability evaluations of ERP systems. However all of these
studies have taken place in industry and not in a higher education environment. The level of
experience of an undergraduate student is not the same as a person in industry, therefore their
perceptions of usability may differ. Existing studies of ERP adoption in the IS curricula are focused
primarily on the pedagogical aspects and have not explored the usability or ease of use of the ERP
system. Empirical research relating to the ease of use or usability of ERP systems is required (Topi et
al., 2005), particularly with regards to the evaluation of the ease of use of medium-sized ERP systems
in educational environments (Hustad and Olsen, 2011).
This paper investigates how IS students experience the usability of a medium-sized ERP system
which forms part of an IS educational program. The investigation aims to propose a set of usability
criteria for evaluating the usability of ERP systems used for instructional purposes and to report on the
usability evaluation of a medium-sized ERP system using these criteria. The usability criteria are
intended to support the identification and classification of usability issues that occur while learning to
use ERP systems in higher education. This paper will contribute to the improvement of the usability of
ERP systems for students who need to use these systems as part of their undergraduate IS courses.
The investigation aims to address the following research questions:
1. What usability criteria and heuristics can be used to evaluate the usability of a medium-sized
ERP system during the learning process?
2. How do students rate the usability of a medium-sized ERP system whilst learning to use it?
3. What features of the user interface of a medium-sized ERP system do students like the least
and which features do they like the most?
This paper is structured as follows: Section 2 will discuss ERP systems and ERP usability criteria and
heuristics used in usability evaluations. Section 3 will explain the research methodology adopted and
the participants involved in the study. The results of the study are presented in Section 4 while
Section 5 analyses and discusses the findings. Conclusions and recommendations are made in
Section 6.
2. Usability criteria for ERP systems used in the learning process
In the late 1990s, many organisations that had not previously adopted ERP systems became
motivated to do so because of the Y2k problem and the introduction of the Surbanes-Oxley Act in
2002 (Vaman, 2007). The Surbanes-Oxley Act legislates that top management are now liable for
certifying the accuracy of financial information (Monk and Wagner, 2009). Several reports have cited
the growing dominance of ERP systems in the business, academic and medical community
(McGauhey and Gunasekaran, 2007; Mohamed and McLaren, 2009). Even small to medium-sized
organisations have implemented ERP systems (Esteves, 2009; Koh et al., 2009). Despite the current
economic and business climate, there is still a positive forecast for the ERP market and according to a
report from Forrester, the global ERP system market will increase to $50.3bn by 2015 (Forrester,
2011).
Shaul (2005) proposes that an ERP system can be classified according to the size of the organisation
that it is designed to support. There are four tiers in this classification with Tier 1, the Enterprise level,
addressing the needs of the largest organisations with incomes of over $200 million (Figure 1). Tier 1
ERP systems include SAP, Oracle, Microsoft and SageERP (SYSPRO, 2010). Vendors have also
introduced medium-sized ERP systems which are specifically designed for organisations in Tier 2
(upper mid-market) and Tier 3 (lower mid-market). Organisations in Tier 2 are usually in the $20
million to $200 million income bracket and ERP systems in this tier include those in Tier 1 as well as
SYSPRO, Softline ACCPAC, SAP Business All-In-One and others. Tier 3 software is designed for
single site organisations of under $40 million income and with five to 30 users. ERP systems for small
to medium organisations are classified as Tier 4 software which are comprised of basic accounting
systems such as Pastel, Accpac and Quickbooks.
.
Figure 1: ERP Systems by Tier (SYSPRO, 2010) {Figure not referred to in the text?}
The global demand for ERP systems has increased the demand for ERP competencies and several
IS education programmes have adopted the hands-on use of ERP systems for instructional purposes
with successful results (Hustad and Olsen, 2011; Wang et al., 2009). Surendran and Somarajan
(2006) report that the market value of students increases when exposed to an ERP system that has a
high market share. A large percentage of studies exposing students to the hands-on use of an ERP
system, reported an improvement in knowledge by the students and positive learning outcomes
(Surendran and Somarajan, 2006; Wang et al., 2009; Winkelmann and Leyh, 2010). However, the
adoption of ERP systems in education programmes is often hindered by the poor usability of complex
ERP systems (Surendran and Somarajan, 2006).
Several studies on the use of ERP systems in industry, report on their poor usability (Singh and
Wesson, 2009; Yeh, 2006) and difficulties with learning to use these systems (Topi et al., 2005).
Similar findings are reported in higher education, where students find that ERP systems are a
challenge to learn and the complexities of the UI often force them to focus on the completion of the
tasks without understanding how the tasks contribute to the underlying business concepts (Wang et
al., 2009; Winkelmann and Leyh, 2010).
Working with medium-sized ERP systems that are less complex than large scale systems such as
SAP can still provide students with an understanding of the basic capability of an ERP system (Ask et
al., 2008; Hustad and Olsen, 2011; Winkelmann and Leyh, 2010). This alleviates the problem of
student frustration encountered while learning the ERP system (Scott and Walczak 2009). The Hustad
and Olsen (2011) study reported that students were more satisfied with the lower learning curve of the
Microsoft Navision ERP laboratory assignments compared to the assignments on the large, complex
SAP system. Microsoft Navision is a Tier 2 ERP system and therefore classified as a medium-sized
ERP system.
The usability of a system has been defined by the International Organisation for Standardisation (ISO)
as the “extent to which a product can be used by specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a specified context of use” (ISO, 1997). This standard
identifies the three criteria of usability as the effectiveness, efficiency and satisfaction with which users
within the organisation are able to accomplish their task before selecting a final solution. Nielsen
(1993) describes usability as: “the quality attribute that assesses how easy user interfaces are to use”.
According to Nielsen (1993) the components (or criteria) of usability are learnability, efficiency,
memorability, errors and satisfaction.
Studies of ERP system usability evaluations propose several usability criteria which can be used in
the evaluation process (Costa, 2010; Van Norren, 2009). Usability criteria can be used to analyse the
user experience and can help reveal patterns that may be hard or even impossible to see (Tullis and
Albert, 2008). A usability criterion can be converted into several related heuristics which are
guidelines or general principles that can guide a UI design decision or that can be used to evaluate a
decision that has already been made (Nielsen, 1993). A literature review of studies of usability
evaluations was undertaken with particular reference to ERP system usability evaluations and the
criteria and heuristics used for these evaluations. Three criteria for testing the usability of open-source
ERP systems in industry were proposed by Costa (2010) are the three proposed by ISO (1997)
namely: effectiveness, efficiency and satisfaction. In another study of ERP systems usability in
industry, Van Norren (2009) used the criteria of learnability, efficiency, memorability, errors and
satisfaction; however the focus of this study was on cultural differences with regards to usability.
A study by Singh and Wesson (2009) identified existing usability issues with a medium-sized ERP
system, SAP Business One, by means of a heuristic evaluation using five categories of criteria,
namely: navigation, presentation, task support, learnability and customisation. The Singh and Wesson
(2009) study was the only study found which proposed and validated usability criteria for medium-
sized ERP systems, and each of the five criteria recommended in this study is supported by several
other studies. Navigation has been reported as a primary design issue in several other ERP usability
studies (Calisir and Calisir, 2004; Mathews, 2008; Surendran and Somarajan, 2006; Topi et al., 2005),
besides the Singh and Wesson (2009) study and can therefore confidently be used as a criterion for
evaluating the usability of ERP systems. Navigation problems have been identified as one of the main
barriers that prevent ERP systems from delivering their potential benefits to organisations (Matthews,
2008). The criterion of Navigation and Access to Information and its related heuristics aim to
determine the ability to identify and access appropriate information, menus, reports, options and
elements accurately and effectively. Presentation of Screen and Output, and its related heuristics aim
to determine the appropriateness of the layout of the ERP system menus, dialogue boxes, controls
and information on the screen for data entry and output generation (Singh and Wesson, 2009). Issues
identified in usability studies of ERP systems relating to presentation include problems with the
complexity of the screen display and problems in understanding and interpreting output from the ERP
system (Costa, 2010; Singh and Wesson, 2009; Topi et al., 2005; Wang et al., 2009; Winkelmann and
Leyh, 2010). The presentation criterion for ERP systems include consideration of how well the visual
layout is designed.
Learnability is the criterion used to determine the degree of effort required to learn how to use the
system efficiently and effectively (Nielsen, 1993; Preece et al., 2006) and can be used as one criterion
to evaluate ERP system usability (Singh and Wesson, 2009; Wang et al., 2009; Winkelmann and
Leyh, 2010). Learnability is one of the most important criteria of usability and refers to the capability of
the system to enable the user to learn to use the application (Nielsen,1993). Customisation relates to
the ability of the ERP system to be customised according to the specific needs of an organisation
(Singh and Wesson, 2009). Customisability is a measure of the extent which the system can be
adapted, either by the user or by the system. Appropriateness of Task Support aims to establish if
there is an accurate alignment between the system and the real world, in order to ensure effective
task support and efficient task completion (Singh and Wesson 2009).
The results of the literature review as well as an analysis of the five criteria proposed by Singh and
Wesson (2009) revealed that three of these criteria are most suited for evaluating the usability of ERP
systems used for instructional purposes in introductory ERP courses, namely navigation, presentation
and learnability (Table 1). These three criteria were validated in the highest number of related studies
in similar environments. A further motivation for eliminating the criterion of task support was due to
the fact that the tasks performed by the students are prescribed by the instructor and are therefore
mandatory. In addition students are novices in terms of business task knowledge so are not qualified
to rate the system in terms of task support. Since customisation tasks are generally not included in
introductory ERP education programmes, the criterion of customisation is also not required for
evaluating ERP systems in education
Table 1: ERP Usability Criteria (Adapted from Singh and Wesson, 2009)
NAVIGATION
Navigation and Access to Information
Information can be easily accessed
Functionality can be found quickly and easily
The user interface supports efficient and accurate navigation of the system
There is a correlation between the searched item and the required item.
PRESENTATION
Presentation of Screen and Output
A user can learn how to use the system without a long introduction
The various functions of the system can be identified by exploration
There is sufficient on-line help to support the learning process
study strategy is therefore a suitable strategy to use for this study where the objective is to evaluate
the usability of a medium-sized ERP system and in particular to identify the features (positive and
negative) of the UI of a medium-sized ERP system.
A single case study approach was used for this research and the case study selected is the Nelson
Mandela Metropolitan University (NMMU) in South Africa, where a Management Information Systems
(MIS) course aims to establish a sound theoretical knowledge of management information systems
and ERP systems, as well as the attainment of transactional competency of ERP systems. The three
core competencies addressed in the ERP section of the MIS course are Business Process
Management (BPM); ERP Theory and Concepts and ERP Transactions. The ERP system which was
adopted into the course for instructional purposes was the medium-sized ERP system, SYSPRO,
which has been identified as one of the best ERP solutions in the Tier 3 category (Kristine, 2011).
Students in the MIS course are required to attend three lectures and four practical sessions where
they perform hands on tasks on the SYSPRO ERP system (Table 2). The practical sessions were one
hour long and took place once a week for three weeks. Four usability questionnaires (P1 – P4) were
administered to participants, one at the end of each practical session in order to obtain quantitative
and qualitative feedback regarding the usability of the SYSPRO ERP system. All four usability
questionnaires comprised of the same set of questions and consisted of two sections. The first section
of the questionnaire required the participant to rate 10 usability criteria on a 5-point Likert scale.
These criteria are grouped into the three usability criteria categories proposed (Table 1), namely:
navigation, presentation and learnability. The second section of the questionnaire included several
open-ended questions, which enabled the users to give feedback on their perceived positive and
negative features of the SYSPRO system.
Table 2: Lectures and Practical Sessions in ERP
Week Lecture Competency Practic Research Instrument
al (Usability
Sessio Questionnaire)
n
Week 1 Lecture 1 ERP Theory and Concepts Prac 1 P1
Week 2 Lecture 2 Business Process Prac 2 P2
Week 3 Lecture 3 ERP Transactions Prac 3 P3
Prac 4 P4
Face validity of the questionnaire was established since the questions were derived from and agreed
on by literature, whilst content validity was confirmed by means of a pilot test where the research
instruments were refined. Cronbach’s alpha coefficients for the summated scores derived from the
four usability questionnaires (P1 – P4) were calculated (Appendix A). Nine of the 10 criteria were
within the acceptable range for Cronbach’s alpha which means the internal consistency is high
(Nunnally, 1978). Learnability in the usability questionnaires P1 ( = 0.55) and P2 ( = 0.6) obtained
values in the range 0.50 to 0.69, the interval deemed acceptable in the early stages of basic research.
The only item which had a low Cronbach’s alpha was presentation in P1 ( = 0.45). The related
results should therefore be treated with caution. One sample t tests were performed (Rosenthal and
Rosnow, 2008) on the usability ratings in order to calculate the significance of positive or negative
scores, where the following interval ranges apply: Negative = [1 to 2.6); Neutral = [2.6 – 3.4]; and
Positive = (3.4 – 5].
The research study, abided with the procedures of the Research Ethics Committee (Human) of the
NMMU and ethical approval for this study was awarded by the NMMU Ethics Committee. In line with
this, the researchers made explicit to the student participants that their interests would be
safeguarded and that they would remain anonymous. The cohort of MIS students that consented to
participate (n = 36) was a convenience sample and are representative of students that typically enrol
in the course every year. The MIS course is a core module in a 3 year undergraduate IS degree. All
the students are full time students with no industry experience. The data for 8% of participants (n = 3)
was eliminated due to missing values, resulting in a sample size of 33 which could be used for
analysis. The majority of participants (79%) fall in the age group 21 to 25 years and are female (Table
3). The fact that the majority of students in the sample were female could be considered as a possible
limitation due to potential gender bias.
Considering the overall usability ratings for all four practical sessions (Figure 2), presentation had the
highest score ( = 3.51), with navigation second ( = 3.44) and learnability lowest ( = 3.33).
Presentation and navigation both had scores in the positive range, whilst the learnability score was in
the neutral range.
Table 4: Positive Usability Criteria for SYSPRO
Usability Criteria Mean t d.f. p-value
NAVIGATION
There is a correlation between the searched item and the required item. 3.83 3.37 25 .002
PRESENTATION
Information provided by the system is timely, accurate, complete and
3.75 3.24 25 .003
understandable.
6. Qualitative results
Qualitative data analysis of the open-ended responses from the usability questionnaires, was
performed using content analysis whereby the text is categorised or coded into themes or categories
(Kolbe and Burnett, 1991). The text responses were analysed and coded according to themes where
the three usability criteria were used as pre-existing (apriori) themes. Within these existing themes,
content was analysed again and in more detail, coded and then similar codes were grouped into sub-
themes. The number of responses per theme was then calculated in order to determine the highest
and lowest frequencies within a theme. The responses related to the question regarding the features
most liked were coded as “positive” responses (Section 4.2.1), whereas the responses to the question
regarding the features least liked were classified as “negative” responses (Section 4.2.2). .
Positive
Users also liked the general structure of the menu which enabled easy navigation and some
examples of responses which related to this theme are:
The menu structure is logically laid out and easy to navigate;
All options (Accounts Payable, Accounts Receivable) are grouped into folders and tree structures,
making navigation of the companies accounting records easy to obtain.
The usability criterion with the second highest count of responses for positive features was
presentation. One aspect of presentation which the users liked was the layout of the SYSPRO
interface. Several participants liked the logical layout and grouping of information and how structured
various parts of the user interface are. One user reported that ‘All information is grouped according to
orders, suppliers and customer information, instead of having it all together.’
The ability to hide information that is not being used was also reported as a positive feature in the
presentation theme. Another user expressed this as follows, ‘Information can be hidden or visible and
as such if I don’t want to see it but don’t want to close it I can just hide it’. Another aspect of the
presentation criterion cited by participants is the attractiveness of the interface. The concept of
attractiveness is similar to the concept of aesthetic and minimalist design identified by Nielsen (1994),
a key criterion for UI design.
In the learnability theme, several users reported that they found SYSPRO’s user interface familiar and
similar to the Microsoft interface and that this assisted them with learning SYSPRO. One user stated
that ‘Menu bars and menu items are easy to understand. They are similar to other familiar
applications' menus’. Familiarity is a factor of learnability and is the extent to which a user’s
knowledge and experience in other real-world or computer-based domains can be applied when
interacting with a new system (Compeau and Higgins, 1995).
Scroll bars
Tabs
Several problems related to the learnability criterion were reported, specifically to the lack of guidance
provided by the system. One participant stated, ‘Difficult to determine where to go and what to do the
first time I used the system’.
9. ERP design guidelines
The results of the usability evaluation of the medium-sized ERP system, Syspro, revealed that overall
the aggregated quantitative usability criteria of presentation and navigation were rated positively.
Learnability was the lowest criteria, but was still not rated negatively as the mean ratings were in the
neutral range. However this could be improved so system developers and designers need to consider
improving the learnability of ERP systems when designing them for instructional purposes. The
highest rated issue related to the finding of required items which suggests that it is critical to consider
this criteria and related heuristics when designing the UI of an ERP system.
Several qualitative negative responses related to the lack of guidance provided by SYSPRO were
received which confirmed findings of a related study reporting that an introductory interface should be
provided by an ERP system to cater for novice users learning to use the system (Topi et al., 2005),
and that an ERP system should provide guidance for novice users in order to improve navigation
(Surendran and Somarajan, 2006).
The most popular features of SYSPRO related to navigation and the main menu design and structure.
The tree structure of SYSPRO’s menu and its drill down feature was cited by several participants as a
positive feature which enabled easy navigation. This confirms the principle that user interfaces should
provide a tree view menu that enables drilling down to specific information very quickly (Shneiderman
and Plaisant, 2005). A tree structure is recommended when a collection of menu items grows and
becomes difficult to maintain under intellectual control. Designers of ERP systems should take into
account the principle that broad and shallow menu structures should be provided, rather than narrow
and deep ones (Norman and Chin, 1988).
The additional frequently cited positive features related to the presentation and layout of the SYSPRO
interface, specifically the menu and the grouping of information. The familiarity of the SYSPRO
interface was also identified as a feature that the participants viewed positively. This supports findings
of related studies (Galitz, 2007; Hustad and Olsen, 2011; Koh et al., 2009) stating that ERP systems
that employ familiar concepts and use a language that is familiar to the user can improve the ease of
learning. It can be deduced that these principles should be incorporated into the design of ERP
systems used for instructional purposes.
The most frequently cited negative features of SYSPRO related to the presentation criterion and was
connected to the cluttered UI as a result of too many menus and windows which caused confusion.
This confirms Shneiderman and Plaisant’s (2005) theory that the incorrect use of tree structures in
menus can occasionally lead to confusion or disagreement. The reported experience of confusion by
users of ERP systems has not been identified by any previously related studies of ERP use in
education, but could however be related to the concept of frustration experienced by ERP users as
reported by Topi et al. (2005). These findings also support other research reporting the problems with
the complexity of the UI and the need to cater for novice users (Topi et al., 2005). This problem could
be avoided by adhering to the user interface design principle of simplicity which states that a dialog
should consist of the minimum and should be natural and logical for the user to use (Nielsen, 1993).
Hidden controls such as scroll bars in several windows in the SYSPRO user interface, was another
frequently reported negative feature. This supports the principle that functionality can be placed in
context-sensitive controls or hidden user interfaces, but that this should not be the only way in which
the function can be invoked (Johnson, 2000). Less visible or hidden methods can be provided, but
these should always be secondary methods, with more visible methods being primary. SYSPRO
designers could therefore investigate other methods of offering the functionality provided by the
hidden controls.
10. Conclusions
The research questions of this paper have been successfully addressed. The results indicate that the
three categories of criteria of navigation, presentation and learnability, together with their related
heuristics, can be used to evaluate medium-sized ERP systems in higher education (RQ1). The
usability of the medium-sized ERP system, SYSPRO was generally rated positively by the students;
therefore the second research question has therefore been answered. Several qualitative feedback
was provided by students in terms of the features they liked they most and those they liked the least,
which provided an in depth analysis of students’ experience of learning to use a medium-sized ERP
system. The third research question relating to the positive and negative features of SYSPRO
encountered by users while using a medium-sized ERP system have thus been successfully
addressed.
The results show that medium-sized ERP systems can be successfully used in education; however
their usability can still be improved further The findings confirm other related studies reporting the
importance of navigation in ERP systems used in industry and in education. Whilst several navigation
problems were encountered, there were also several aspects of the user interface which the students
liked, such as the tree view and menu structure. Several students cited ‘confusion’ due to the
complexity of windows. The importance of using a system that is familiar was also reported. Therefore
several practical, key contributions have been made in this paper for designers of ERP systems. ERP
designers should take into account the results of the analysis of the usability related responses in
order to assist them when designing an ERP system which is intended for instructional use. The
reviewing case study results reported in this paper has provided issues relating to the usability of ERP
systems that might be of interest to a wide range of users and technology experts. Several design
guidelines for ERP systems for instructional purposes were provided (Section 5). The most important
guideline relates to providing guidance for novice students and the need for an introductory interface.
Other guidelines proposed relate to the use of familiar interfaces, languages and concepts; the use of
broad and shallow menu structures; the use of hidden controls as secondary methods and the
avoidance of clutter and complexity in ERP UIs. The contributions can therefore facilitate the
improvement of the usability of ERP systems. ERP educators can also benefit from the results
presented in this paper by taking into account the proposed criteria and guidelines when planning
which ERP system to use and when evaluating potential systems. In addition they should consider the
benefits of implementing a medium-sized ERP system which may be less complex than a large ERP
system.
The insights provided by this case study provide important contributions to Human Computer
Interaction (HCI) researchers, ERP researchers and other researchers and can assist with a deeper
understanding of other similar cases or situations. The use of mixed methods data analysis using both
quantitative and qualitative data analysis in a case study strategy provided insight into understanding
how users complete tasks while learning to use an ERP system that would not have been possible
with only a positivistic approach using purely quantitative data analysis.
This study formed part of a larger study which investigated the competencies required for ERP
graduates as well as the different learning approaches. Future research could explore various ERP
system alternatives for ERP educational purposes and do a comparison of the usability of large ERP
systems, medium-sized ERP systems and simulated ERP systems. The results of this study have
highlighted the need for an ERP system used for instructional purposes, which is not as complex as
existing industry ERP systems but can still provide the required competencies. Additional research on
the usability of medium-sized ERP systems is recommended which can explore the design of an ERP
system that can support teaching and provide improved user guidance, especially for novice users.
References
Ask, U., Juell-Skielse, G., Magnusson, J., Olsen, D.H. and Päivärinta, T. 2008.Towards the third-wave of higher
enterprise systems education: Three Scandinavian cases. In Proceedings of the 31st Information Systems
Research Seminar (IRIS 31), Åre, Sweden, 10-13 August.
Beer, S. 2010. IT Salaries to rise as CRM, CMS and ERP skills shortages loom. [online] Available at:
http://www.itwire.com/it-people-news/recruitment/40982-it-salaries-to-rise-as-crm-cms-and-erp-skills-
shortages-loom?start=1 [Accessed 6 March 2011].
Benbasat, I., Goldstein, D.K. and Mead, M. 1987. The case research strategy in studies of information systems.
MIS Quarterly, 11(3), pp. 369-386.
Boyle, T. A. and Peslak, A. R. 2010. An exploratory study of the key skills for entry-level ERP employees.
International Journal of Enterprise Information Systems, 6(2), pp. 1-14.
Compeau, D.R. and Higgins, C.A. 1995. Computer self-efficacy: Development of a measure and initial test. MIS
Quarterly, 19(2), pp. 189-211.
Costa, C.J. 2010. Testing usability of ERP open source systems. In Proceedings of the OSDOC'10. Lisbon,
Portugal, 8 November 2010.
Esteves, J. 2009. A Benefits Realisation road-map framework for ERP usage in small
and medium-sized enterprises. Journal Of Enterprise Information Management, 22(1/2), pp. 25-35.
Foltz, C., Schneider, N., Kausch, B., Wolf, M., Schlick, C. and Luczak, H. 2008. Usability Engineering.
Collaborative and Disributed Chemical Engineering, LNCS 4970, pp. 527-554.
Forrester. 2011. ERP market to grow to $50.3bn in 2015: Forrester. [online] Available at:
http://enterpriseapplications.cbronline.com/news/erp-market-to-grow-to-503bn-in-2015-forrester-060511
[Accessed 8 August 2011].
Galitz, W.O. 2007. The Essential Guide to User Interface Design. 3rd edition. Indianapolis: Wiley Publishing, Inc.
Hustad, E. and Olsen, D. H. 2011.Teaching Enterprise Systems in Higher Education: The Learning Context
Triangle. In Proceedings of the 8th International Conference on Enterprise Systems, Accounting and
Logistics (8th ICESAL 2011). Thassos Island, Greece, 11 - 12 July 2011.
Ifinedo, P. and Nahar, N. 2009. Interactions between contingency, organizational IT factors, and ERP success.
Industrial Management & Data Systems, 109(1), pp. 118-137.
ISO. 1997. ISO9241-11: Ergonomic requirements for office work with visual display terminals (VDT's). Part 11-
guidelines for specifying and measuring usability. International Standards Organization (ISO). Available at:
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=16883 [Accessed 7 April
2010].
Johnson, J. 2000. GUI Bloopers: Don'ts and do's for software developers and web designers. San Francisco:
Morgan Kaufmann Publishers.
Kristine, H. 2011. Best ERP solutions based on Tier. erp.com. [online] Available at: http://www.erp.com/section-
layout/337-erp-software/8912-best-erp-solutions-based-on-tier.html [Accessed May 2011].
Koh, S. C. L., Gunasekaran, A. and Cooper, J. R. 2009. The demand for training and consultancy investment in
SME-specific ERP systems implementation and operation. International Journal of Production Economics,
122(1), pp. 241-254.
Kolbe, R.H. and Burnett, M.S. 1991. Content-analysis research: An examination of applications with directives for
improving research reliability and objectivity. Journal of Consumer Research, 18(2), pp. 243-250.
Law, C. C. H., Chen, C. C. and Bruce, J. P. W. 2010. Managing the full ERP life-cycle: Considerations of
maintenance and support requirements and IT governance practice as integral elements of the formula for
successful ERP adoption. Computers In Industry, 61(3), pp. 297-308.
Lee, A.S. 1989. A scientific methodology for case studies. MIS Quarterly, 13(1), pp. 33-50.
Magal, S. R. and Word, J. 2012. Integrated Business Processes with ERP Systems. Hoboken NJ: Wiley.
Matthews, D. 2008. Usability as an ERP selection criteria. MIT Sloan Management Review. [online] Available at:
http://download.ifsworld.com/home/if1/page_341/usability_as_an_erp_selection_criteria.html/ [Accessed
February 2009].
McGauhey, R. E. and Gunasekaran, A. 2007. Enterprise Resource Planning (ERP): Past, Present and Future.
International Journal of Enterprise Information Systems, 3(3), pp. 23-35.
Mohamed, S. and McLaren, T. S. 2009. Probing the Gaps between ERP Education and ERP Implementation
Success Factors. AIS Transactions on Enterprise Systems, 1, pp. 8-14.
Monk, E. F. and Wagner, B. J. 2009. Concepts in Enterprise Systems, 3rd edition. Boston: Cengage Learning.
Ngai, E. W. T., Law, C. C. H. and Wat, K. F. T. 2008. Examining the critical success factors in the adoption of
enterprise resource planning. Computers in Industry, 59, pp. 548-564.
Nielsen, J. 1993. Usability Engineering. San Francisco: Academic Press.
Nielsen, J. 1994. Ten Usability Heuristics. [online] Available at: http://www.useit.com [Accessed 18 September
2009].
Norman, K.L. and Chin, J.P. 1988. The effect of tree structure on search in a hierarchical menu selection system.
Behaviour & Information Technology, 8(2), pp. 51-65.
Nunnally, J. 1978. Psychometric Theory. 2nd edition. New York: Mc-Graw Hill.
Preece, J., Rogers, Y. and Sharp, H. 2011. Interaction design: Beyond human computer interaction. 3rd edition.
Chichester: John Wiley & Sons Ltd.
Rosenthal, R. and Rosnow, R. 2008. Essentials of behavioural research: Methods and data analysis. Journal of
Psychology, 75(3), pp. 349-362.
Rothenberger, M. A., Srite, M. and Jones-Graham, K. 2010. The impact of project team attributes on ERP system
implementations: A positivist field investigation. Information, Technology and People, 23(1), pp. 80-109.
Scholtz, B., Cilliers, C. and Calitz, A. 2011. Critical competencies for South African ERP Consultants.
International Business Conference (IBC 2011), Mauritius. September 2011.
Scott, J.E. and Walczak, S. 2009. Cognitive Engagement with a multimedia ERP training tool: Assessing
computer self-efficacy and technology acceptance. Journal of Information and Management, 46 (2009), pp.
221-232.
Seethamraju, R. 2007. Enterprise Systems (ES) Software in Business School Curriculum - Evaluation of Design
and Delivery. Journal of Information Systems Education, 18(1), pp. 69-83.
Seymour, L., Dawood, M., Mandengu, T. and Mokobori, L. 2006.The Contribution towards Student
Competencies: An Analysis of an Information Systems Business Process and ERP Course. In Proceedings
of the 2006 South African Computer Lecturers' Association (SACLA) Conference. Cape Town, South Africa.
Shaul, C. 2005. ERP: What tier are you in? [online] Available at: http://www.erpandmore.com [Accessed 26 May
2009].
Shneiderman, B. and Plaisant, C. 2005. Designing the user interface: Strategies for effective human-computer
interaction. 4th edition. Maryland: Pearson Addison-Wesley.
Shtub, A. 2001. A framework for teaching and training in the Enterprise Resource Planning (ERP) era.
International Journal of Production Research, 39(3), pp. 567-576.
Singh, A. and Wesson, J. 2009. Evaluation criteria for assessing the usability of ERP systems. In Proceedings of
the South African Institute of Computer Scientists and Information Technologists (SAICSIT) 2009. Vaal
Dam, South Africa, October 2009.
Surendran, K. and Somarajan, C. R. and Holsing, D. 2006. Incorporating ERP into the MIS Curriculum: Some
insights. Information Systems Education Journal, 4(25), pp. 3-12.
SYSPRO. 2010. SYSPRO. [online] Available at: http://www.syspro.com [Accessed 22 May 2010].
Theling, T. and Loos, P. 2005. Teaching ERP Systems by a multiperspective approach. In Proceedings of the
Americas Conference on Information Systems (AMCIS). Nebraska, USA.
Tullis, T. and Albert, B. 2008. Measuring the User Experience. Burlington: Morgan Kaufmann Publishers.
Topi, H., Lucas, W. and Babaian, T. 2005. Identifying usability issues with an ERP implementation. In
Proceedings of the International Conference on Enterprise Information Systems. Miami, USA, 24-28 May.
Vaman, J. N. 2007. ERP in Practice: ERP strategies for steering organizational competence and competitive
advantage, New Delhi: Tata McGraw-Hill Publishing Company.
Van Norren, D. 2009. Cultural Differences in Usability of Enterprise Resource Planning Systems. Unpublished
masters thesis, Faculty of Humanities. Tilburg University, Netherlands.
Wang, M., El-Masry, E-H.E. and Zhang, X. 2009. Assessments and Outcomes of an ERP/SAP Fundamental
Course. Issues in Information Systems, X(1), pp. 109-114.
Winkelmann, A. and Matzner, M. 201009. Teaching Medium-Sized ERP Systems - A problem-based learning
approach. Information Systems Development, pp. 891-901.
Winkelmann, A. and Leyh, C. 2010. Teaching ERP systems: A multi-perspective view on the ERP system
market. Journal of Information Systems Education, 21(2), pp. 233-237.
Yeh, J. Y. 2006. Evaluating ERP performance from a user perspective. In Proceedings of the IEEE Asia-Pacific
Conference on Services Computing (APSCC'06). IEEE Computer Society.
Yin, R.K. 2003. Case study research: Design and methods. 3rd edition, Thousand Oaks, CA: Sage Publications.
Appendix B: Descriptive Statistics for the Usability Criteria for all Practical
Sessions
NAVIGATION
Functionality can be found quickly and easily 26 3.23 0.73 1.50 3.38 4.50
PRESENTATION
The visual layout is well designed 26 3.25 0.70 1.50 3.25 4.50
LEARNABILITY
A user can learn how to use the system without a long introduction 27 3.22 0.88 1.75 3.25 4.50
The various functions of the system can be identified by exploration 27 3.42+ 0.80 1.25 3.50 4.50
There is sufficient on-line help to support the learning process 27 3.31 0.63 1.75 3.25 4.50
+
Positive range