CHAPTER 5 Data Integration and Testing Once you have the shell of your database, and have started

writing reports, you will need data in the system to validate that you have a good build and good performance. Data is what this is really all about. Good data in the system will make all the difference. The better the data quality coming in, the better information you will have, and the better decisions you will make. It drives everything in the tool. ut first you have to get the data into the application. There are many kinds of data you can load into !"#. It is most common to load numeric values like what would be e$pected for the trial balance, or supplemental data. This data is primary to the system, in that it is always needed to report the financials. There is data that supports the financials, and that is supporting data. "or e$ample, there is cell te$t that could be loaded to a given cell. There are documents that could be attached to cells. Intercompany Transaction data, %ournal &ntries, and 'nnotations are all supplemental to the trial balance data. There are, of course, data values that we cannot load into !"#. They are often requested, but the system currently will not support it( for e$ample, transaction data, date) oriented data, or unstructured data. #ost pro*ects will need to pull data from many sources. These many sources are often very different, each with different formats and fields. In fact, the more sites and entities you have, the more types of data feeds you will see. There are many different ledgers, offline worksheets, data marts, and warehouses. There are many options for getting the data in, with many kinds of bells and whistles. +ome can automate( others can help with metadata, and allow drilling into the source data. &ach tool offers different benefits, and some tools offer overlapping features. ,ith so many options, it is hard to see what the right tool is, and when to use it. "ortunately, Oracle offers tools to help with getting data into !"#. +ome of the options are simple, like +mart -iew, ,eb "orms, or the use of a flat file to load data directly. These offer quick and simple ways to get data into the application, but they often require some manual manipulation or data entry to get it to work. If you are looking for full) featured automation, validation, and auditability, then you are looking at one of the feature)rich tools. #any of these tools are often referred to as &T. tools. This stands for &$tract, Transform, and .oad. The e$tract phase involves having a hook into the source systems, and identifying what data it should pull. Then the data must meet an e$pected pattern or structure. If not, the data may be re*ected entirely or in part. The transform phase maps the data to the destination, through mapping or rules. The load phase writes the data to the destination, !"# in this case. ,hen !"# was first released by !yperion +olutions, the only ways they provided for you to load data were through a flat file, using the &$cel add)in or !yperion 'pplication .ink /!'.0. 1sing &$cel to load data presents problems. There are too many opportunities to modify the data and mapping. There are really no controls or reliable ways to back it up. These &$cel files often need to be saved on a server, where anyone with access can see all the data loaded. !'. was powerful, but it was a platform. It was a tool that needed to be implemented for each client from scratch. That was a ma*or issue. &ach solution was unique to the team that designed and developed it. That meant when you had a ma*or change or update, you needed to get the people who built it to come back and help you. The other issue is that it could be as comple$ as you wanted to make it, and it was made comple$. !yperion did offer Data Integration #anager /DI#0 for a short time. This was built on the Informatica tool. DI# was not widely adopted for two reasons2 !yperion acquired 1pstream +oftware and the ,eb.ink product, and after the Oracle acquisition of !yperion, ,eb.ink was later renamed "inancial Data 3uality

#anagement. 4eople did not really have time to learn DI# before it was hurried out the door for the two products people use now. 1pstream offered the very successful product ,eb.ink, which provided e$actly the sort of end user5oriented tool that !"# customers wanted, whereas DI# requires a very high level of technical e$pertise to develop and run. Oracle then later replaced DI# with the standard across all Oracle tools, Oracle Data Integrator /ODI0. ,eb.ink was rebranded as "inancial Data 3uality #anagement /"D#( yes, only three letters0 and Oracle added Oracle Data Integrator /ODI0 to the suite of !yperion applications. ,hile you still have the simple options of forms, &$cel add)ins, and flat files, "D# and ODI offer feature)rich options. The Right Tool for the Right Job "D# and ODI together offer a full solution for those customers using !yperion usiness 4erformance #anagement solutions. &ach product has strengths that should be leveraged to provide customers with a best)of)breed solution for data integration and transformation. The best solutions are never one or the other, but the right mi$ that ma$imi6es reliability and auditability and improves the data quality. ut which one should you use and when7 "D# is a financial data integration platform that delivers a web)based data collection solution that is focused on both finance and end users. This framework provides prebuilt process and logistical support as well as visibility to all aspects of the data integration process. 8ey +arbanes)O$ley controls and processes are 9out of the bo$: in "D#. If you want a packaged solution, this is the answer. 'lso, data quality and integrity is ensured through data validation and quality checking against target systems. "D# has a great user interface, which makes it a great tool for end users, whereas ODI is not. "D# is designed for use over the web, which helps to bring in data files from all over the world. "D# is not as powerful as the best &T. tools when it comes to large volumes of records to load, whereas ODI is. "D# offers a process and reporting that helps get data loaded by reporting the details of the mapping and data file process. The financial requirements defined by +arbanes)O$ley require increased audit and process controls delivered by "D# 9out of the bo$.: "D# also has tremendous fle$ibility via - +cript to solve the most comple$ data movement and translation problems. "or companies with multiple locations, multiple source systems, and multidimensional mapping requirements, "D# will help drive down the cost and time of implementation cycles of !"# as well as speeding adoption and increasing ;OI. Typically, "D# is used for loading !yperion &nterprise and !yperion "inancial #anagement( however, "D# can also be an e$cellent solution for !yperion &ssbase and !yperion 4lanning. ODI is a data integration and business process automation tool that dramatically reduces the time and e$pense of integrating source data with the !yperion usiness 4erformance #anagement suite. Designed specifically for IT or application power users, ODI enables end)to)end connectivity with one or more data sources, <== percent business process automation, and broad data source connectivity including purpose)built adapters for transaction processing applications from vendors, such as +'4, 4eople+oft, %. D. &dwards, and +eibel. ODI is built for each client>s needs for that pro*ect, offering a framework to build integration solutions that suit e$act business needs. ?ou can build either tool to achieve complete process automation. 'dapters are available for each tool to integrate with all !yperion packaged and tailored)made applications to support data and metadata e$change. The choices are not one or the other. In fact, the best solutions will include the best features of each tool. Organi6ations are looking to leverage the integration strengths of ODI along with the 9packaged:

out)of)the)bo$ functionality of "D#. &$amples includes organi6ations with a central &;4 for 1.+. operations and a variety of G. systems across the globe. "D# can call ODI integration capabilities through &;4i to read in the ledger data while utili6ing "D#>s Data 4ump to provide a seamless method for loading pure G. source files that come from all over the world through 6ero)footprint web interfaces. Then you can take advantage of functionality providing key +arbanes)O$ley controls including2 financial data transparency, end)user visibility, and audit trail to source data, automation, batch processing, process monitoring, mapping consistency, and data quality checks. ,hen both tools work together, they can offer a better solution than either alone. Data Flow &ach company has its own set of databases and systems that also impact the decision of what tool or combination of tools work best. The first principle, and the most important principle you should adhere to, above anything else, is to reduce the number of reconciliation steps. +ome companies have a data mart they want to use as the source of all the data they have, and they need the consolidation tool. Or, for whatever reason, they do not want to source the data from the ledger. If that company understands that they are giving up some functionality, such as the ability to drill into the data they are loading, it is not the worst thing they could do. 'ssume you have several feeds coming into the data mart. Then you want to have those feed directly into !"#. @ow you have created many feeds that must be reconciled, and !"# must be reconciled to the mart. There is a cost to that. If the drill)through functionality is not required, or valued, then isn>t it a better option to load directly from the data mart7 I would still e$plore sourcing directly from the ledger. One should never dismiss functionality, or disregard a clear product direction. !owever, when a company forces a consolidation tool into a data flow, and it creates places where you have to do additional reconciliation, to make sure that the data does in fact tie out at each spot, then that creates work that is not value added. .inear data flows that do not double back on to other systems are the easiest to maintain. The best systems are easy to e$plain and transparent. Aomple$ data flows that double back and load back into something else create timing issues, mapping problems, and other issues. &ven if you can automate the validation of the data between the systems, you can create problems. 'utomation sometimes creates a sense of security, which allows mistakes to flow from one system to another. 'nother consideration before you decide on a tool2 ?ou should look at the data to be loaded. !ow is the data that will be loaded going to be dispersed in your environment7 Is it centrally located or is it decentrali6ed7 Aentrali6ed data environments would have only one or two general ledger systems, where most data to be loaded into !"# resides. In this situation the data loading, mapping, and validation can be controlled by a centrali6ed team. Decentrali6ed data environments have many general ledger systems or sources of data, typically spread out across many remote locations. Decentrali6ed data often creates the impossibility of having a centrali6ed group manage mappings. The ownership of mapping and validation has to be pushed to the sites. The central core application team has to focus on managing the effort. 's this chapter covers the tools you can use to load data, it will highlight where some tools handle each of these functions better than the others. Oracle Hyperion Financial Data Quality anage!ent

Oracle !yperion "inancial Data 3uality #anagement /"D#0 is a packaged data transformation tool that feeds financial data to !"#, 4lanning, and &ssbase. "D# is a great tool for accountants and end users. It has some nice features that help with audit and reconciliation. It has the ability to see a history of the mapped data. "D# helps users, administrators, and auditors to investigate, identify, and correct errors. There are controls and validations you can build in to make sure the data being loaded is good. ?ou can get adapters to load to !"# 4lanning and &ssbase, or pull from many types of sources. The "D# tool is best suited for supporting decentrali6ed data, although it can be used for centrali6ed sources. This tool uses a very intuitive interface that makes it easier for nontechnical users to load and map files. The interface is 6ero)footprint5based, meaning there is nothing to install on users> machines. This simple interface allows users to load, map, and validate data in a uniform and controlled process, by stepping through each step of the process. 'nyone who has used this tool will tell you how simple it is to see the fish work their way through loading the data. ,hat are the fish, you say7 The fish walk users through the steps they need to do to get data into the system. The fish are icons used to symboli6e the movement of data 9upstream,: which is a play on the original company>s name. "igure B)< shows the "inancial Data 3uality #anagement web interface and the fish. The basic "D# process includes si$ steps2 <. Import source data. C. -alidate source data against mapping tables. D. &$port source data to a target system. E. Aonsolidate target system data. B. -alidate target system data. F. ;eview and validate internal financial control. ,hy should you use "D#7 The value of "D# is that it is a packaged solution. 'll of the data is loaded into a central repository that contains all source data and maps. This allows you to drill down and drill through, providing an audit trail to the data loaded into the system. ,ithin this database, "D# archives all of the source files, error logs, and load files, so you can go back to them at any time and see what was loaded and by whom. ?ou can load many periods and automate the loading into batch *obs. ?ou can create control questions and use the certification feature that helps you comply with sections D=C and E=E of the +arbanes)O$ley 'ct. 1sers can load additional data, view spreadsheet *ournals, and see the data broken out by the source in the reports. There are full sets of prebuilt reports that allow you to monitor the loading of the data into your applications. ?ou can see *ust about everything you need for "D# in these reports. "D# uses maps and tables to translate a source dimension value to !"# dimensions. #aps are created for each !"# dimension /'ccount, &ntity, IA4, .ine Item Detail0. ,ithin each dimensional map, there are four map typesG&$plicit, etween, In, and .ike. These maps run in a specific order. The order is <. 'ccount

C. &ntity D. IA4 E. .ine Item Detail +ince you have this sequence, you can map other dimensions based on the results of another dimension>s mapping. "ewer source dimension combinations result in fewer maps being required. The result is faster "D# data processing. ,ithin a dimension map, the map types process in the following order2 "# E$plicit ' single source dimension is mapped to a single target dimension. &$plicit maps are the most efficient and straightforward type, and hence the fastest. !owever, &$plicit can result in increased maintenance. %# &etween ' range of source dimension members is mapped to a single target. ,ildcards are not supported in etween maps. '# (n ' noncontinuous list of source dimensions that maps to a single target dimension. "or e$ample, H===, H==<, and H==C map to revenue. These are the slowest of the mapping options, so they shouldbe used sparingly. )# *i+e +ource dimensions are mapped using wildcards in place of actual characters. "or e$ample, all accounts that begin with H are mapped to ;evenue /HI J ;evenue0. .ike maps support only two types of wildcards, asterisks /I0 and question marks /70. The question mark represents only one character. 'n asterisk is used to represent one or more characters. +o 9HI: would mean any number of characters beginning with H, while 9H7: would mean any two characters beginning with H. "or all of these reasons, it is very unusual to see !"# without "D#. It provides so many benefits to getting data into !"#, in such an easy way.