Journal Online

Continuous Auditing Reexamined

Norman Marks was the leader of internal audit functions at major US and global corporations for more than 15 years prior to joining SAPs GRC solutions team in late 2008. He has held management positions in IT auditing, including 11 years with Coopers & Lybrand (where he coauthored the global firms approach to the audit of IT general controls). Marks spent several years in IT management, with responsibility for IT security, network design, quality assurance and more. He is a recognized international leader in the theory and practice of internal auditing, and is on the editorial board of four periodicals.

A few assertions have been made in papers and presentations about continuous auditing that merit closer examination: 1. Continuous auditing is the application of automated tools.1 2. A recent research paper by Gartner implied strongly that monitoring transactions provides assurance that controls are in place and operating effectively.2 3. A number of software vendors have implied, if not stated, in presentations that effective continuous auditing can be achieved through the implementation of their out-of-the-box automated testing routines. These assertions do not, in fact, stand up to close examination. In the following sections, each will be discussed and a different view explained and justified. CoNtiNuous AuditiNg is AN AutoMAted teChNique Deloitte, in an Institute of Internal Auditors (IIA) webinar in July 2008, said, Continuous auditing and monitoring solutions are technology-enabled, detective controls utilized to actively monitor controls, transactions and configurations. On its Information Technology Center web site, the American Institute of Certified Public Accountants (AICPA) reported: Continuous auditing defines the technologies and processes that allow an ongoing review and analysis of business information on a real-time basis. Continuous auditing will require specialized skills of audit personnel to monitor information electronically and incorporate the use of intelligent agents, computer modeling and other software tools. Is continuous auditing only an automated technique? To answer the question, it is first necessary to agree on what continuous auditing is. The IIAs Global Technology Audit Guide (GTAG) on continuous auditing defines it as any method used by auditors to perform audit-related activities

on a more continuous or continual basis. ISACA defines it as an approach that allows IS auditors to monitor system reliability on a continuous basis and to gather selective audit evidence through the computer. Can all controls be tested using technology? The answer is no. Take the example of a physical inventory count of a warehouse. It is possible to develop a test to confirm that a count has been recorded (e.g., by testing for the execution of the inventory reconciliation program), but it is not possible to use an automated test to verify that the count was of good quality. Many years ago, an IT auditor for what is now PricewaterhouseCoopers visited the head of internal auditing for a refining company. Outside his office, several junior staff were hunched over calculators. He explained that because of a history of problems (including fraud), his staff reviewed every bank reconciliation. They were continuously auditing the bank reconciliations. More recently, the chief audit executive (CAE) for a large global manufacturing company attended the companys disclosure review committee meetings. The committee had oversight responsibility for the system of disclosure controls. The CAE also sat in on the chief financial officer (CFO)s quarterly calls with the division controllers, where he questioned them on the financial results, trends, variance explanations and any potential financial reporting issues. The attendance of the head of internal auditing at these meetings provided him with assurance that (a) they were held, and (b) the control procedures that were completed at these meetings were operating properly. This is also an example of continuous auditing of these controls. There are many examples of continuous auditing procedures: attending IT operations meetings where the results of the prior nights processing are discussed, participating in employee training on building security or evacuation procedures, or completing the annual employee certification of the code of conduct.
ISACA JOURNAL VOLUME 1, 2010

All of these are valid continuous auditing techniques. They are used to provide assurance that controls are operating effectively on a more continuous basis. In fact, the design of an effective continuous auditing program requires: 1. Identifying the controls to test 2. Understanding the systems, data and processes in place and assessing the available testing techniques. Determining whether automated techniques can be used involves understanding: Whether the data are available to test (e.g., in a data warehouse) The tools that can be employed (e.g., enterprise business intelligence applications, automated testing products, stand-alone audit query software) The level of assurance on the effectiveness of controls provided by testing data (more on this in the next section) 3. Designing and implementing the continuous tests In most cases, if a continuous auditing program is to provide a reasonable level of assurance on all the key controls required to manage a business risk (including entity-level controls, manual and automated business process controls, and IT general controls), it has to include a combination of manual and automated tests. CoNtiNuous exAMiNAtioN of dAtA for ANoMAlies ANd exCeptioNs is A WAy to test CoNtrols Internal auditing is all about obtaining assurance that controls are effective in managing risk. Knowing that there is an effective system of internal controls provides the board and management with a forward-looking perspective; they have a reasonable level of assurance that activities will be carried out as they expect, resulting in reliable financial reporting, compliance with applicable laws and regulations, and operations that are run efficiently. Testing data for anomalies and exceptions is an after-thefact detective technique. (There are a few software solutions, such as SAPs Process Control product, that can test data within the enterprise resource planning [ERP] system. However, almost all products extract the data for subsequent query and analysis.) While testing data will detect errors and potential frauds, care must be taken before assuming that clean data imply effective controls. To illustrate the point, an individual may consider whether his/her home has been broken into in the last year. Presumably, 2
ISACA JOURNAL VOLUME 1, 2010

that has not happened, but the fact that there has not been a break-in (i.e., the data are clean) does not mean that he/ she remembered to lock the door every time he/she went out (i.e., that the control was in place and operating as intended). There are a number of situations during an audit where there are no errors, but the control is not operating at all. For example, the bank reconciliation is completed properly, but the reviewer did not examine itjust added his/her signature. Designing a continuous auditing test that verifies the control is in place requires deeper thought than simply testing the affected transactions. Sometimes, an automated test needs to be supplemented by manual observation or auditor inquiry. The level of controls assurance that testing data will provide should be considered when following the steps discussed previously to design the continuous auditing program. If it is not sufficient, alternative or supplementary procedures should be included. effeCtive CoNtiNuous AuditiNg CAN Be AChieved through out-of-the-Box AutoMAted testiNg routiNes Vendors of continuous auditing solutions typically provide a great deal of valuable content, including predefined tests, that can be included in a continuous auditing program. However, organizations have different business risks, systems, processes and controls. An effective continuous auditing program: 1. Is designed using a top-down approach that starts with the business risks that will be addressed, then identifies the controls that manage the risks, and finally designs a continuous auditing program that provides reasonable assurance that those controls are in place and operating effectively 2. Involves a variety of testing techniques, including both manual and automated tests, able to provide assurance on the combination of entity-level, manual and automated business process, and IT general controls required to manage the major business risks This top-down, risk-based approach avoids one of the major pitfalls of simply installing and implementing an outof-the-box solution: being very busy testing data and perhaps controls, but not necessarily achieving the primary objective of internal auditing (i.e., providing assurance that the system of internal controls effectively manages the organizations more significant risks within organizational tolerances).

A differeNt pArAdigM: CoNtiNuous risk ANd CoNtrol AssurANCe One of the lessons of the recent financial meltdown, shared by both financial and nonfinancial organizations, is that organizations need to understand the risks they face, and those risks can change alarmingly quickly. Internal auditors can Risks can change and should adopt a risk-centric alarmingly quickly. mindset3 and evaluate the effectiveness and contribute to the improvement of risk management processes.4 Rather than providing risk and control assurance5 through periodic audits, todays technology enables internal auditors to provide a high level of risk and control assurance continuously. Internal audit will provide its customers, the board of directors and executive management assurance that the organizations risks are subject to appropriate and effective processes, including related systems of internal control. The assurance will be enabled primarily through continuous risk and control monitoring and auditing. Continuous risk and control assurance (CRCA) is far more than an application of continuous auditing or monitoring; it is a top-down model (figure 1) that starts with enterprise goals and objectives, moves on to risks to the objectives and the controls required to manage the risks, and includes the mining of data that can provide indicators of risk and control health. The models foundation is built upon the more significant strategic and operational goals of the enterprise. Achievement of the goals and objectives is measured through key

figure 1CrCA Model

Organizational Goals and Objectives Risks Continuous Risk Monitoring Continuous Control Auditing Continuous Assurance Dashboard Continuous Data Mining/Auditing and Fraud Detection Alerts Key Performance Indicators Key Risk Indicators

Controls

Enterprise Data

On-demand Data Mining

Data Warehouse

performance indicators (KPIs). The CRCA model includes the monitoring of KPIs, as the failure to achieve organizational objectives is often the result of poor risk management or control performance. Risks to the achievement of those goals and objectives are then identified. Continuous risk monitoring, generally by a risk management function and not by internal audit, ensures that the CRCA program is focused on the more significant risks to the enterprise, which may change rapidly. Internal audit is a consumer of the monitoring but not responsible for its performance. Management of those risks (i.e., risk responses) is enabled by controls. The model includes the continuous auditing of the key controls required to manage risks within organizational tolerances, usually performed by internal audit and other assurance providers, sometimes by operating management. Some controls are difficult to test directly through automated routines, and the continuous examination or testing of data may provide a reasonable level of assurance. The data mining may be directly against information within the organizations applications, or indirectlyafter extraction to a data warehouse (either an existing corporate data warehouse or one developed specifically for this purpose). Fraud management is built into the CRCA program. Fraud risks are identified together with other enterprise risks; the controls required to manage fraud risks are assessed and tested together with other key controls; and data mining techniques can be used to test data and identify potential fraud situations. When assurance is continuous, information on the health of risk management and related key controls needs to be continuously available to stakeholders. The model includes the capability for continuous reporting through dashboards or similar tools. It also includes more immediate alerts signaling the need for a response to a spike in risk levels, an adverse incident, a potential control failure or a data anomaly. On-demand data mining enables an intelligent response and investigation of data anomalies, control failures, etc. The same tools that provide continuous control and data auditing will generally also support additional data analytics that provide further insight into the problem.

ISACA JOURNAL VOLUME 1, 2010

CoNtiNuous AuditiNglessoNs of A CAe By Lynn Fountain, president, ExpertGRC LLC and past vice president of risk assessment and audit services, Aquila Inc. Software vendors and technology advocates have the audit world entranced with the notion that continuous auditing can be completed only through the utilization of automated techniques. Although it can be a strong enabler to the process, technology may not be necessary in all situations and at all times. The audit team of a publicly traded company in a highly regulated environment embarked on the challenge of further evaluating the concept of continuous auditing. The team did so at a critical juncture in the companys history; as a direct report to the CEO, the CAE was aware of the companys strategic evaluations for the potential sale of the company. As a multistate utility, the approval process could extend over 18 months. Also, at any point in the process, the sale could be nullified by regulatory disapproval. The sale negotiations were taking place at the same time as submission of the annual audit plan to the audit committee. Knowing the potential roadblocks for performing routine internal audits, it was necessary to find alternative methods to provide risk assurance in an everchanging business environment. The audit team had made previous attempts at continuous auditing through the use of ACL data analytics. The team members often found themselves stalled by the mechanics and information overload. They were not focused on the strategy of the process and how it met the business objectives and corporate strategy. With the knowledge of the impending sale announcement, the CAE instructed the auditors to build a complete methodology and framework to enable them to perform continuous auditing on risk areas related to the corporate strategy. Maintaining a risk-based approach, a framework was developed that identified process areas to be examined, specific control objectives that were of risk concern, testing methods (including manual, automated and observation), risk indicators and reporting processes. When this framework was in place and the organizations risk indicators and parameters were established, the team was able to define which processes should be technology-enabled and which processes would need to be monitored through other methods. Upon announcement of the sale of the company, the selected processes and control objectives for risk areas that would be of highest concern during a long period of regulatory approvals and a changing internal environment were refined. Technology was used to identify anomalies in accounts payable, payroll, travel and entertainment, and other data-rich areas, while other observations or manual efforts were used to continuously evaluate areas where segregation of duties could become an issue as individuals left the organization. The program required the CAEs continual attention (it was in use for over a year until the companys sale in July 2008) to changes in the business and related risks. The team learned from this application of a highly touted concept that the success of continuous auditing may not hinge as much on technology as it does on the establishment of a solid framework for the programs execution coupled with direct linkage with the business strategies, objectives and risks.

suMMAry Individuals interested in building a home for their family do not go to the local hardware store and buy the best-looking tools and wood, then go back to the lot and start putting nails in the wood. Instead, they first determine their needs and agree on a design. Only when they know what they plan to achieve and the steps involved are they able to consider acquiring the necessary tools and materials. The same principle applies to continuous auditing techniques. Auditors should not go to their favorite software vendor and buy their automated testing solution until after they know what they want to accomplish. Auditors must identify the risks for which they will provide assurance, identify the key controls that manage the risks, determine the best way to test those controls on a continuing basis, and finally select the tools. Figure 2 provides a high-level view of the process. The sidebar provides an example case of a successful continuous audit implementation. The only step not shown in figure 2 is that of continuous improvement. It is achieved not only through periodic assessment of the program, but also through the continuous monitoring of risk and the addition of continuous control testing of new or heightened risks.

figure 2Building a Continuous risk and Control Assurance program

What risks will be addressed through continuous assurance? Significant Risks

What controls are relied on to manage risks within tolerances?

Perform contiuous risk monitoring.

Key Risk Indicators

Entity-level Controls

Manual and Automated Business Process Controls

IT General Controls Continuous Risk and Control Assurance Dashboards

Manual and Automated Business Process Controls

What strategy will be used to test and provide continuous assurance for these controls?

Control Health Reports

ISACA JOURNAL VOLUME 1, 2010

eNdNotes 1 Wikipedia, Continuous Auditing, http://en.wikipedia.org/ wiki/Continuous_auditing 2 Caldwell, French; Paul E. Proctor; Continuous Controls Monitoring for Transactions: The Next Frontier for GRC Automation, Gartner. Continuous controls monitoring for transactions (CCM-T) is an emerging governance, risk and compliance (GRC) technology that monitors ERP and financial application transaction controls to improve financial governance and automate audit processes. This is later clarified: Transaction monitoring functions automatically, periodically imports transaction data from ERP and financial applications, and applies a set of predefined audit analytics to identify control exceptions. 3 PricewaterhouseCoopers (PwC), Internal Audit 2012, 2007, www.pwc.com 4 IIA, International Standards for the Professional Practice of Internal Auditing, www.theiia.org 5 Op cit, PwC

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal. Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors content. 2010 ISACA. All rights reserved. Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25 per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited. www.isaca.org
ISACA JOURNAL VOLUME 1, 2010