International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Cobit 4.1: A Maturity Level Framework For
Measurement of Information System
Performance (Case Study: Academic Bureau at
Universitas Respati Yogyakarta)
Herison Surbakti
Universitas Respati Yogyakarta
Indonesia
Abstract - Universitas Respati Yogyakarta requires methods
and structured approach in its evaluation, especially in
academic departments. This should to be done to assess the fit
between institutional objectives with management that has
been applied. COBIT is a framework used to assess, measure
and control the performance of institutions in the
management of IS/IT. COBIT is also accepted and
harmonized by its users, because the framework is built from
the goal, rules and institutional policy where all processes are
analyzed by looking at the alignment between the objectives to
RT
be achieved by the procedures/policies implemented by the
institution. In this study the author uses COBIT (Control
Objectives for Information and Related Technology) version
4.1 on the domain Planning Organization (PO). The results of
IJE
the study conducted performance measurements are made in
the form of academic information system analysis, mapping
the level of maturity and recommendation for Universitas
Respati Yogyakarta, which is expected to be a management
model for other institutions.
Keywords
COBIT, measurement of performance, audit, CSF, KGI, KPI
1. INTRODUCTION
Some companies do not hesitate to invest their share in
the field of Information Technology (IT), although the
investments make enormous drain on the budget. This is done as
an effort to get the convenience and benefits of the use of IT,
which is expected to help the performance of the company to
conduct a competitive business strategy. However it turns out, the
investment made by the outcome often is not generated.
Assessment and evaluation of investments that have
been issued for the implementation of IT is proper to be
considered. Based on some research explained that the company
has begun to realize and start doing performance measurement
and evaluation. To make the use of performance measurement
and management of IT, then The IT Governance Institute (ITGI)
as an institution conducting the IT governance arrangements that
have standardized tools or frameworks is widely used in the world
including COBIT, ITIL, COSO, ISO, and so on [2],[4],[5],[9].
COBIT is a reference method/framework for
measurement and control of information technology. COBIT
framework is a standard that is considered the most complete and
IJERTV3IS080897
(This work is licensed under a Creative Commons Attribution 4.0 International License.)
Vol. 3 Issue 8, August - 2014
thorough as IT audit framework as developed based on the
rules/procedures of internal company/institution where COBIT is
used, so the time will be measured in accordance with the
conditions, rules, procedures and norms that work in the
company. COBIT has also been developed on an ongoing basis by
professional NGOs auditors spread throughout much of the
country, where in each state builts a relationship that can manage
these professionals.
Performance measurement in academic information
system at the Universitas Respati Yogyakarta that utilizes IT as a
means of supporting the process is expected to support the
management of the education process. For example at the
beginning of the selection for new students, the learning process
which is done, lectures supporting components such as the
method used, the curriculum and other provisions such as faculty,
students, facilities, other facilities until the graduation of students
who need to be evaluated in order to produce quality and good
service and competitive education.
1.1. Problems
1. There has been no measurement of the performance model of
academic information system at the Universitas Respati
Yogyakarta.
2. COBIT as a framework used as a reference for measuring the
performance of academic information system at the
Universitas Respati Yogyakarta.
1.2. Limitations
Limitation of the problems in this study is:
1. This study refers to the standard of COBIT 4.1 (Control
Objectives for informatian and related technology) made by
the IT Governance Institute (ITGI) an institution that manage
and organize IT governance.
2. COBIT domain used is on Planning Organization (PO).
1.4. Research Objectives
The purpose of this study is:
1. Make a model of the academic information system refers to the
COBIT framework according to the characteristics of
Universitas Respati Yogyakarta.
2. Gain the insight about the performance of academic
information system at the Universitas Respati Yogyakarta.
3. Perform analysis and assess the fit between policies on
academic standards at the Universitas Respati Yogyakarta
implementation, it is used as a reference for the evaluation of the
existing academic system at the Universitas Respati Yogyakarta.
www.ijert.org 999
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

2. LITERATURE REVIEW

2.1. Information Technology Architecture

The Information Technology Architecture (ITA) is
described through a set of views, each from the perspective of a
different stakeholder. An individual view captures items
meaningful to the stakeholders as elements and their
interrelationships expressed in a standard form, the structure of
the view, and the views correlation with other views. Typically,
groups within the organization possess a mixture of Commercial
Off-The-Shelf (COTS), frameworks and custom-developed
legacy applications as well as data stored in databases and
directories. In such an environment, functionality and data
embedded in one application cannot be easily merged with
functionality and data provided by another. Such an endeavor
usually requires the creation of a yet another separate application.
Further, each of these applications relies on a proprietary client
for user interaction [10].

Figure 2.2 Relationship of data, information, and knowledge

RT
IJE

Figure 2.3 Description of the data, information and knowledge

Figure 2.1 Conceptual View of the architecture (EOHHS CTO
Organization Information Technology Architecture, Version 2.0, 2007)
2.2. Information System
It conceptually describes the data objects, events,
activities, and transactions, which have no meaning or effect
directly to the user. Information is data that has been processed in
such a way that increases user knowledge [10],[1],[3],[4]. While
knowledge is a combination of instincts, ideas, rules, and
procedures, that drives the actions or decisions. The relationship
between data, information and knowledge can be mapped in
Figure 2.2, while the data, information, and knowledge can be
described in the form of a pyramid, as shown in Figure 2.3.
2.3. Academic Information System
To achieve the goal of the learning process that is done,
it needs a system to regulate the course of the process of the
learning system, from start of new admissions, teaching and
learning activities, faculty, students, lecturer's method used, the
curriculum, the process of filling up the study card, and the
graduation evaluation process. This is what is known as the
academic system [4],[9]. Academic System is made with the
purpose of regulating the learning process to fit the expected
goals, in accordance with the vision and mission of educational
institutions that shelter, and also in accordance with the rules
instituted university education.
2.4. COBIT Framework
COBIT Framework is an IT governance framework
aimed at management, IT service staff, department control, and
audit function more critical to the business process owner. COBIT
Framework as a framework for the management needs for
measurement and control of information technology provides the
tools to measure the ability of information technology that will
continue to be developed. COBIT framework is created to serve
as a reference to exercise control over information technology and

IJERTV3IS080897 www.ijert.org 1000

(This work is licensed under a Creative Commons Attribution 4.0 International License.)
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

can ensure confidenciality, integrity and availability of the data From the data obtained, and then the researcher
[5],[6],[7]. performed the steps in the performance measurement system of
The results obtained from the IT Governance is used as academic information such as explained figure 3.1.
a reference or management guidelines, one of which is COBIT
which has a measuring tool such as maturity models, CSF, KGI
and KPI. COBIT has a measurement indicator management of
information technology in business processes. The relationship
between the measuring tool set is described in Figure 2.4.

Figure 2.4 Model CSF, KPI dan KGI

3. RESEARCH METHOD
The research method used in this study:
1. Studying the source literature of COBIT Framework, audit
trails, and academic information system.
2. Data collection from internal documents regarding
institutions such as vision, mission, goals, IT architecture,
organizational structure, master plan development, including
RT

IT management policies.
3. General identification of the learning process that is carried
out at the Universitas Respati Yogyakarta.
IJE

4. Perform an analysis of the weaknesses, strengths,

opportunities and challenges of the institution (SWOT
Analysis).
5. Direct observation of the academic activities to gain
referring to the direct point of the matter.
6. Perform analysis of the academic information system that
refers to the COBIT framework using a questionnaire.
7. Conducting interviews to related parties such as bureau of
academic chief, chairman of the study program, IT staff and Figure 3.1 Stages Performance and Measurement of Academic
administrative staff, and faculty. Information System
8. Perform data analysis.

3.1 Stages of Performance Measurement of Academic 4. IMPLEMENTATION

Information System
The research method used is performance
measurement system with reference to the academic information
system and considering some points as follows:
1. Internal Institute Environment
1. Vision, Mission, and Institution Goals
2. Academic Institutions Activity Cycle
3. Strategy and Scope of Business (Business Scope)
4. Distinctive Competence
5. Development Master Plan
6. Planning factors
7. Strategic Planning
8. Architectural Institute of Information Technology
9. Integrated Academic Information Management
System of Institutions
10. Integrated Systems Students
4.1 Performance Measurement of Academic Information
System Framework Based on COBIT
To perform the management and measurement of
performance against the academic information system, the
framework which is used as a reference or guideline is COBIT
Framework which includes the maturity model, CSF, KPI and
KGI. In general, IT arrangement in Universitas Respati
Yogyakarta aims to drive, ensure, and control the institutions in
order to achieve goal, namely to reduce the risk factors and make
improvements in all aspects so as to support the institution's
performance [4],[8].
Figure 4.1 describes the framework for the
establishment of IT management adopted from COBIT domains
directly with Planning Organization (PO) as a focus of study.

IJERTV3IS080897 www.ijert.org 1001

(This work is licensed under a Creative Commons Attribution 4.0 International License.)

Figure 4.1 IT Institutional Management Framework Based on COBIT's
PO domain (ITGI, Management Guidelines, 2008)
RT
4.2 Performance Measurement Model of Academic
Information Systems
IJE
Academic information system procedures and policies
in its practice have clear rules, standardized and should be
informed. The determination of the performance achievement
indicators (strategic objectives) using CSF, KPI and KGI are used
to achieve this. Figure 4.2 is a planning model for the
management of academic information system at the Universitas
Respati Yogyakarta with reference to the COBIT Framework.
IJERTV3IS080897
(This work is licensed under a Creative Commons Attribution 4.0 International License.)
International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014
Figure 4.2 Managing and Organizing the design of academic
information system
4.3 Performance measurement process on the domain of
Planning and Organizing
The measurement process is expected to manage IT
resources and achieve strategic goals and objectives such as
effectiveness, efficiency, confidentiality, integrity, availability,
compliance and reliability of the academic management
information systems, so this stage is the translation of the vision
and policy institutions. To make the process of performance
measurement at the PO domain, the used of indicator known as
KGI (Key Goal Indicators) and KPI (Key Performance
Indicators), this both indicators related to the CSF (Critical
Success Factors) because CSF was raised from the need for
business done by management and if not done then it will hamper
the pace of organization.
To map the institutional objectives and business
requirements, the process can be done by translating and
identifying indicators associated with the domain Planning
Organization, such as: Establishing the performance of each
process PO, Assign KGI, KPI Assign, Assign CSF, and Maturity
models. The measurement process is shown in Figure 4.3.
www.ijert.org 1002

Figure 4.3 Performance Measurement Process of Academic Information
System Management
4. Descriptive Maturity Model and Measurement
Techniques
Descriptive measurement techniques are made by the
nominal size to sort objects from the lowest to the highest, these
measurements only give the order by rank. Measurements were
carried out directly from values that refers to the value of the
existing sorting in maturity models as show in table 4.1.
Table 4.1 Maturity Model (ITGI, Management Guidelines, 2008)
RT
Level Name When
0 Non-Existent Management has not recognised the need for
a process for defining service levels.
IJE
Accountabilities and responsibilities for
monitoring them are not assigned.
1 Initial/Ad-Hoc There is awareness of the need to manage
service levels, but the process is informal
and reactive. The responsibility and
accountability for defining and managing
services are not defined. If performance
measurements exist, they are qualitative
only with imprecisely defined goals.
Reporting is informal, infrequent and
inconsistent.
2 Repeatable There are agreed-upon service levels, but
they are informal and not reviewed. Service
level reporting is incomplete and may be
irrelevant or misleading for customers.
Service level reporting is dependent on the
skills and initiative of individual managers.
A service level co-ordinator is appointed
with defined responsibilities, but limited
authority. If a process for compliance to
SLAs exists, it is voluntary and not enforced.
3 Defined Responsibilities are well defined, but with
discretionary authority. The SLA
development process is in place with
checkpoints for reassessing service levels
and customer satisfaction. Services and
service levels are defined, documented and
agreed-upon using a standard process.
Service level shortfalls are identified, but
procedures on how to resolve shortfalls are
informal. There is a clear linkage between
expected service level achievement and the
funding provided. Service levels are agreed
IJERTV3IS080897
(This work is licensed under a Creative Commons Attribution 4.0 International License.)
International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014
to, but they may not address business needs.
4 Managed Service levels are increasingly defined in the
system requirements definition phase and
incorporated into the design of the
application and operational environments.
Customer satisfaction is routinely measured
and assessed. Performance measures reflect
customer needs, rather than IT goals. The
measures for assessing service levels are
becoming standardised and reflect industry
norms. The criteria for defining service
levels are based on business criticality and
include availability, reliability, performance,
growth capacity, user support, continuity
planning and security considerations. Root
cause analysis is routinely performed when
service levels are not met.
5 Optimised Service levels are continuously re-evaluated
to ensure alignment of IT and business
objectives, whilst taking advantage of
technology, including the cost-benefit ratio.
All service level management processes are
subject to continuous improvement.
This simplifies the process of calculating and mapping the
maturity model. These measurements were made to map the
position of IS governance maturity model into the existing, so that
from the model that will be scalable of the IT management in
planning and organizing domain occupies a level determined in
accordance with criteria that are owned. From its placement then
it can be used to evaluate candidates for the management for
further improving and enhancing the performance of the
management of the existing IS governance in the institution.
4.1 Summary Results of Audit Implementation Design
Model
Recapitulation of this implementation shows all the
results of data from questionnaires filled out by respondents from
different levels of management. The questionnaires recapitulation
use descriptive measurement techniques. This is a standard
indicator outcome of determination recap associated with
quantitative data expressed in simple calculations such as the total
value of the whole, the index, average, and percentage. The
answers given by the respondents will have scores that is
equivalent to each of its maturity level. The total respondent is 25
and the recapitulation of PO results in table 4.3.
The respondents calculation summary per PO (Planning
Organization) domain by using descriptive calculation formula
obtained results as shown in Table 4.3 and 4.4.
www.ijert.org 1003
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

Table 4.3 PO Domain Questionnaire recapitulation

5. CONCLUSION
PO TOTAL INDEX % LEVEL
From the results obtained it appears that the institution
PO-1 56 2.24 44.80% 2 has a different level for each PO domain. In general, institution
occupies level 3, which means that the institution has procedures
PO-2 46 2.19 61.33% 3 in management, has been communicated and documented for each
element in the institution. However, the implementation is still
PO-3 49 2.33 65.33% 3 highly dependent on existing human resources willing to perform
the procedure or not. That existing procedures are still limited to
PO-4 56 2.67 74.67% 4 the formalization of implementation available, and management
needs to improve planning and organizing.
PO-5 44 2.10 58.67% 3 The discrepancy between the technical implementation
of management policies with existing will be reduced by the
PO-6 57 2.71 76.00% 4 control and measurement refers to guidelines such as COBIT
Framework KPI, KGI, CSR and maturity models in COBIT
PO-7 66 3.14 88.00% 5 Framework for monitoring ranging from policy-setting;
monitoring is being conducted at the time, until the goal is
PO-8 59 2.81 78.67% 4 achieved.

PO-9 61 2.90 81.33% 4 6. REFERENCES

PO-10 56 2.67 74.67% 4

[1] Alter, Steven, 1992, Information System A Management

TOTAL 713 33.53 920.80% 36
Perspective, Fourth Edition, Prentice Hall Inc.
[2] Garca, Victoriano Valencia., Vicente, Eugenio J. Fernndez, Dr.,
MEAN 71.3 3.353 92% 3.6 and Aragons, Luis Usero, Dr., 2013, Maturity Model for IT Service
Outsourcing in Higher Education Institutions, (IJACSA)
International Journal of Advanced Computer Science and
Table 4.4 Percentage scale at the level of maturity Applications, Vol. 4, No. 10, 2013.
[3] Gottschalk, Petter, 2012, Knowledge driven service innovation and
RT
Scale Level Maturity Level management : IT strategies for business alignment and value
creation, ISBN: 1-4666-2512-0, 978-1-4666-2512-9, IGI Global.
0 17% 0 Non-Existent [4] Henderi, 2010, Good IT Governance: Framework and Prototype for
Higher Eduation, Creative Communication and Inovative
IJE

18 33% 1 Initial/Ad-Hoc Technology Journal vol 3, no.2 ISSN: 1978-8282.

[5] ISACA, 2006, Integrating COBIT into the IT Audit Process
34 50% 2 Repeatable (Planning, Scope Development, Practises), IT Governance Institute.
[6] IT Governance Institute, 2007, COBIT 4.1, Framework Control
51 66% 3 Defined Objectives Management Guidelines Maturity Models, IT
Governance Institute, ISBN 1-933284-72-2.
67 83% 4 Managed [7] ISACA, 2006, IT Governance Global Status Report.
[8] Law, M. Averill, dan Kelton, David W., Simulation Modelling and
84 100% 5 Optimised Analysis, Second edition, McGraw-Hill., Singapore, 1991.
[9] Maria, Evi, 2012, The Measurement Of Information Technology
Performance In Indonesian Higher Education Institutions In The
Context Of Achieving Institution Business Goals Using COBIT
Based on the results from Table 4.3, for each process in Framework Version 4.1 (Case Study : Satya Wacana Christian
the PO domain, it obtained graphs as in the figure 4.4 below: University, Salatiga), Journal of Arts, Science & Commerce, E-ISSN
2229-4686, ISSN 2231-4172.
[10] Skoczylas, Robert., Sevier, Raoul., Garden, Martin., Snyder, Jason.,
2007, Commonwealth of Massachusetts, 2007, EOHHS CTO
Organization Information Technology Architecture, Version 2.0.

Figure 4.4 Measurement graphs in maturity model

IJERTV3IS080897 www.ijert.org 1004

(This work is licensed under a Creative Commons Attribution 4.0 International License.)