Informatica CRC Generation Function INFORMATICA CRC GENERATION FUNCTION

The Informatica CRC algorithm generates a unique serial number for every data row that can be extracted from the source system. This function takes set of string, numeric or date fields as input and generates a unique number for the data set as its output. Every time the CRC function is called with this input data set, it would generate the same unique number. This CRC function can be used to compare instances of data to be loaded to the data warehouse over different days to decide, if the corresponding row in the target warehouse needs to be updated or the input row simply ignored.

EXAMPLE USAGE SCENARIO

Let us take the example of a scenario, where customer information from a source system is extracted everyday and loaded to the data warehouse. Let us assume that the customer information comprises of the Customer number, Customer Name, Customer Address, Customer City, Customer Phone number and Customer email. On Day 1, we would be extracting this information from the source system. While inserting these rows to the target data warehouse table, we would call the CRC function with all the customer related information as input data. This function would then return a unique number based on the input data. This unique number would also be written to the target warehouse table as an additional column on the inserted rows (the table schema needs to include that column). On Day 2, let us assume that the Customer City information changed in the source system and the changed row is once again extracted. While loading the changed data to the data warehouse, we would compare all the columns of the customer row that was extracted from the source today with all the columns that already exist for that customer row in the target warehouse to identify if the row really needs to be updated. Instead of doing a column by column comparison using AND logic, we could re-generate the unique number using the CRC function with todays extracted data as in the input and compare it with the CRC number that already exists on the customer row loaded to the data warehouse on Day 1. If the CRC numbers do not match, the row that exists in the data warehouse needs to be updated with the changed information (Customer City) in addition to updating the CRC number column. If the CRC numbers match, the input row should be discarded as no change has been detected using the CRC number logic.

HOW TO USE THE CRC GENERATION FUNCTION

The CRC generation function is available in an external file (Ex: DLL in the case of Windows). This file needs to be called using an External Procedure (EP) transformation in Power Center. Currently this function supports up to 40 input fields. The following are the steps to be followed to use the CRC generation function:

Informatica Developer Network

http://devnet.informatica.com

Informatica CRC Generation Function

Step 1 Create a reusable external procedure transformation. Enter Input ports 1 to 40 and one output number port.

Step 2 Register the DLL or the external file for the external procedure function. Please make sure that the module name is entered as the file name and procedure name is abucrc32calc. Also define the location where the external file would be present.

Informatica Developer Network

http://devnet.informatica.com

Informatica CRC Generation Function

Step 3 Once this function has been defined, it could be used in the mappings in the following manner (example).

SOURCE FILE NAMES

OS Windows NT/2000 HP Unix AIX Sun SOLARIS File Name CRC32.dll libcrc32.sl libcrc32shr.a libcrc32.so.1

INFORMATICA DEVELOPER NETWORK http://devnet.informatica.com

Informatica continues to provide interoperability APIs to enable third parties to create add-on solutions and services for Informatica customers. The Informatica Developer Network helps third parties efficiently leverage this open architecture. It provides a Web-based community for sharing technical knowledge and software and a global forum for nearly 10000 developer members in well over 80 countries.

CRC GENERATION FUNCTION FEEDBACK

Please e-mail devnet@informatica.com with your feedback on the CRC Generation Function.

Informatica Developer Network

http://devnet.informatica.com