Documente Academic
Documente Profesional
Documente Cultură
FUNCTIONAL DETAILED
DESIGN V0.2
Abstract
This document describes the detailed design for the MDM Reference Data solution
RSA – MDM REFERENCE DATA Functional Detailed Design v0.2
Table of Contents
1 Document Overview..............................................................................................................................................3
1.1 Purpose.......................................................................................................................................................................... 3
1.2 Revision History ............................................................................................................................................................. 3
1.3 Document Approvers .................................................................................................................................................... 3
1.4 Document Reviewers..................................................................................................................................................... 3
1.5 Referenced Documents ................................................................................................................................................. 4
1.6 Acronyms ....................................................................................................................................................................... 5
2 Overview ...............................................................................................................................................................7
2.1 Data Insight Architecture............................................................................................................................................... 7
2.2 Scope ............................................................................................................................................................................. 9
2.3 AGENT.......................................................................................................................................................................... 10
2.4 RDM ............................................................................................................................................................................. 10
3 Interface Detailed Design ................................................................................................................................... 11
3.1 User Interfaces ............................................................................................................................................................ 11
3.2 Data Model .................................................................................................................................................................. 11
3.3 RDM SCREENS.............................................................................................................................................................. 13
3.4 Screen Examples .......................................................................................................................................................... 14
3.4.1 Process Screen ................................................................................................................................................. 14
3.4.2 View Screen(OOTB) .......................................................................................................................................... 15
3.4.3 Tasks Screen(OOTB) ......................................................................................................................................... 16
3.4.4 Authorize/Rollback Screen ............................................................................................................................... 16
3.5 Reference Data ............................................................................................................................................................ 18
3.5.1 Key Considerations........................................................................................................................................... 18
3.5.2 Process ............................................................................................................................................................. 19
3.5.3 Input File .......................................................................................................................................................... 21
3.5.4 Output File ....................................................................................................................................................... 21
4 RDM Integration Job .......................................................................................................................................... 22
5 MI Reporting ...................................................................................................................................................... 23
6 AuDIT Logging and Error Handling ..................................................................................................................... 23
6.1 Audit ............................................................................................................................................................................ 23
6.1.1 Audit Table ....................................................................................................................................................... 24
6.1.2 Audit Log File .................................................................................................................................................... 24
6.2 Errors ........................................................................................................................................................................... 24
6.2.1 Message Error Tables ....................................................................................................................................... 24
6.2.2 Error logs .......................................................................................................................................................... 25
7 Security ............................................................................................................................................................... 25
7.1 Access Controls ............................................................................................................................................................ 25
7.1.1 Network And Platform Secruity ....................................................................................................................... 25
7.1.2 Talend SQL Server ............................................................................................................................................ 25
7.1.3 Talend Log files................................................................................................................................................. 25
7.2 Data Security ............................................................................................................................................................... 25
8 Availability .......................................................................................................................................................... 26
9 Requirements Traceability Matrix...................................................................................................................... 27
3.4.1 & 3.5.1 ......................................................................................................................................................................... 29
3.2 & 3.5.2 ............................................................................................................................................................................ 29
Modified: 08/02/2018 01:11 1
Copyright ©2015 Accenture. All Rights Reserved.
RSA – MDM REFERENCE DATA Functional Detailed Design v0.2
3.5.2 30
1 DOCUMENT OVERVIEW
1.1 Purpose
The purpose of this document is to provide a detailed design to component level for the MDM Reference Data
solution. This document extends the MDM Solution Analysis Specification document which provides an
overarching description of the MDM solution and requirements.
Name Role
Ian Kenealy Design Lead
Dom Attwell / Jide Okuwa Data Architect
Gillian Tomlinson CDO
Susan Binns Service Architecture
Tony Prikockis Product Owner
Alistair Goodall Security Lead
Name Role
David Green Business Analyst
Ian Elms Data Modelling Architect
Chris Darragh Data, Analytics & MDM Delivery Lead
Richard Walsh E2E Architecture
Jazz Khan / Paul Dobson Infrastructure
Jon Lillis Infrastructure
Jorge Orts Integration
For assistance with access to JIRA, please reach out to Theo Muzangaza (theo.muzangaza@uk.rsagroup.com)
and/ or David Bodle (david.bodle@uk.rsagroup.com).
0KT%20from%20Bernard/Blueprint%20Feb%2004%20v5.6.pptx?We
b=1
1.6 Acronyms
Acronym Definition
2 OVERVIEW
This document is a technical design for the MDM Reference data solution. It will provide the development team
guidance on what is to be built to component level and include any business logic that must be implemented. It
is also useful reading for anyone developing or supporting the downstream applications which are part of the
e2e Galaxy initiative.
The inputs into this design are the RDM requirements and the MDM SAS (Solution Analysis Specification). This
design should be read alongside both of the afore mentioned documents
Access Layer
NBS View Home D4P Extract
MI MDM Legacy
Analytics
Warehouse
Data Vault AIS MI
CDW
Customers:
No integration PL Quote MI 350k - More Th>n Motor
with the Legacy 1.2m - More Th>n Pet & Home
Data Lake Customer MDM systems inside 20m - Holders of lapsed policies
the dotted & enquirers
Customers: 44m – From Experian Consumer
border. They
Galaxy Home, migrated on renewal from DLG View
are shown for
context.
CSAT NPS
Operational Data Hub
FDS
Telephony
API API
Biztalk CAL SIL
Sources
Agency
(position on
diagram TBC)
Key:
Title: Data Insight Architecture for Galaxy Contributors:
Unchanged Component Changed Component Legacy
Scope: Galaxy RSA: Dom Attwell, Tony Prikockis, Ian Dawson
Level: Level 0 Accenture: Ian Elms, Chris Darragh, Sandeep Raju,
Version: 0.7 Required for Home Only New Component Subramanya Hanuman
A : Day 1 + 12 months Addition
Note: This diagram is currently under review with RSA architecture and subject to change until it has been
approved.
The diagram below describes the key layers of the MDM solution and the main building blocks within each of
the layers that are being delivered.
The RDM FDD’s will provide coverage of all the blocks listed in the block diagram. The coverage of work
described in this design document has been highlighted in the block diagram shown below:
This FDD will describe the design for storing reference data at a central location for the Galaxy Home Insurance
business.
2.2 Scope
This section defines the reference data categories which are in scope of Galaxy. The categories are divided into
Integration, MI, MDM and NBS scope. These have been signed off by Product owner.
2 Integration HearAboutUs
3 Integration ReasonForCancellation
4 Integration ListedBuilding
5 Integration/MDM/NBS MaritalStatus
6 Integration PolicyTransactionReason
7 Integration PolicyTransactionType
8 Integration PolicyTransactionMigration
9 Integration PremiumPaymentMethod
10 Integration ProductBrand
11 Integration ProductName
12 Integration PropertyType
13 Integration Status Type
14 Integration Quote Type
15 Integration Channel
16 Integration/MDM/NBS Title
17 Integration CampaignCode
18 Integration PartnerBranchCode
19 Integration/MDM County
20 Integration Nations
21 Integration AffinityCustomerSegment
22 MI MTA Reasons
The current RDM solution has been designed in line with the in scope reference data categories as use cases, but
the capability allows for additional reference data categories to be introduced by RSA. The whole process of
adding and updating category/reference value has been described in section 3.4.1 and 3.5.2.
2.3 AGENT
Agent data will be mastered in RDM so that there is a single source of this information available for reporting
purposes across all systems in the estate.
Agent scope includes RSA agents only. Mastering of NBS agents in RDM is out of scope because due to data
privacy concerns NBS will not share information about its agents.
It will be considered just like any other category in RDM and will be created/updated by uploading fixed format
csv input file. The process is mentioned in section 3.4.1 and 3.5.2.
2.4 RDM
RDM solution provides following capabilities:
Building capability for creating, updating and disabling reference data in Reference Data Model through
delimited csv file.
Creating capability of using Talend customized Forms and Talend DSC by Data Office stewards to
approve/reject an operation on reference data, view reference data and dependencies using ootb
functionality and data governance process to manage consistency across all systems.
Providing capability for MI solution to interact with RDM and pull reference data for reporting.
CATEGORY
PK Category_ID
Category
SYSTEM
Description
PK System_ID
Creation_DT
System_Name
Last_Update_DT
Creation_DT
Effective_Start_DT
Last_Update_DT
Effective_End_DT
Effective_Start_DT
Authorized_DT
Effective_End_DT
Active
Key
Approval_FL
Classification
CATEGORY_SYSTEM_RELN
Key
REFERENCE_VALUES
PK CAT_SYS_REL_ID
PK Ref_ID
Category_ID
CAT_VAL_ID
System_ID
CAT_SYS_REL_ID
Creation_DT
Ref_Values
Last_Update_DT
Creation_DT
Effective_Start_DT
Last_Update_DT
Effective_End_DT
Effective_Start_DT
Authorized_DT
Effective_End_DT
Publish_FL
Active
Active
Authorized_DT
Total_Levels
Parent_Ref_ID
Level_Num
CATEGORY_VALUES
PK CAT_VAL_ID
Category_ID
Value
Description
Creation_DT
Last_Update_DT
Effective_Start_DT
Effective_End_DT
Authorized_DT
The field level detail is presented in the Reference data model spreadsheet presented below.
Reference Model
specsv0.2.xlsx
The data has been classified as ‘Confidential’ for all attributes in the data model. A definition of this classification
has been provided in section 7.2 of this document.
Note – Any Partner will be considered just like our native source systems and categories specific to partner will
be stored the same way as Galaxy categories in RDM.
Super user/Manager
1. Will use DSC to approve/reject changes to reference data triggered by Data Stewards.
2. Will also manage the priorities of the tasks raised by Data Stewards based on created dates in
DSC. The Super user will be able to see all tasks/changes and their effective dates so that he/she
can prioritize task based on approaching effective date. Based on this super user will have a high
level overview of all open tasks and their deadlines.
Data Stewards
1. Will create, disable and update reference data by uploading csv’s for multiple changes from a
customized Bonita Form. This will trigger an integration job to populate RDM tables and will
send email informing changes and category level file(s) to all-consuming systems like DC, CRM
and BizTalk etc.
2. Will use DSC to view RDM data and its dependencies in a hierarchical manner.
3. Using DSC to roll back the changes in RDM and informing all the source systems through email.
4. Authorize the change to all systems via email to all consuming systems mentioning the effective
start date for that change.
PROCESS
Category
Item System
Reference Value
Create
Action
Disable
Update
Name
Impacted CRM
Systems MDM
DC
Biztalk
After the Steward clicks submit button, the integration job will be triggered to consume the uploaded
file in RDM
Please note based on the relationship you have chosen, only entities following this relationship will be
available to select in pivot field.
Below screen shows a task ‘Category_Create_Title’ created by ‘administrator’ user which is in ‘new’
status.
The below screen shows how a Data Steward can authorize a change from DSC.
After hitting save, the DSC shows the task status as ‘Resolved’.
3.5.2 Process
1. Whenever any system wants to create/update reference data/category they will inform Data Office
about the change via email.
2. Data Office will raise a CR and will approve or reject it.
3. If the CR gets approved, then a nominated data steward will either create, update or disable the
reference data.
4. The data steward will upload a csv file with the new/updated reference categories/values through the
Bonita Form mentioned above which in turn will call a Talend integration job to insert/update data in
RDM.
5. Based on Approval_FL set in input file, the change will either be committed or marked for approval. If
the change is marked for approval, it will be committed in RDM only after the Super user approves the
change.
6. Steps related to different scenarios are mentioned below:
Rollback
This scenario will occur if any of the consuming systems does not implement the changes within time
window given to them.
Changes will be been done in RDM first through integration job. If Publish_FL flag is set as ‘Y’ in
input file, an email will be sent to all consuming systems including BizTalk describing the change,
category xml file and time window in which they have to implement those changes.
If Data Steward does not receive confirmation from any of the consuming systems till a specific
date which will be before effective start date, data steward will not authorize the change and
hence will trigger rollback through DSC as mentioned in section 3.4.4 and all-consuming systems
will be notified about the rollback. It will be source systems responsibility to roll back the
changes.
No
Auto
Update
Flagged for
Reference Data
Approval?
RDM
Yes
Trigger
Yes
Yes
Yes
End
CR raised
Data Office
Updates
title_input.csv
AgencyCountryCod Title.xml
e.xml
Errors
tFileInputDelimited tFileOutputDelimited
9
1 4 5 7
tDie and
tXMLMap tXMLMap tLogCharacteristics tStewardshipTaskOutput
2 3 6 8
MDM Server
The table below describes the data flow and the components illustrated in the diagram above.
Flow # Component Description
tFileInputDelimited This component will read the csv file uploaded on server.
1 tXMLMap The csv file read by tFileInputDelimited is sent to the tXMLMap
component.
This component maps, transforms and route data. It maps the csv format
into the Data Model format in the MDM server. It will parse the fields,
validate the fields, apply business logic/ transformation and map to the
data model structure.
2 tMDMOutput The tMDMOutput component writes to the MDM database.
3 tMDMInput The tMDMInput component reads from the MDM database.
4 tFileOuputDelimited The component will create xml output file at category level.
5 tXMLMap Audit and errors information from the MDM components are handled by
(Errors) the tXMLMap component where they output into the Audit/ Error tables
in the MDM database. The component maps the messages into the
format required by these tables.
6 tMDMOutput The tMDMOutput component writes to the MDM database.
Messages (audit logging) and errors are logged into the Audit tables in
the MDM database.
7 tDie and Process flow 7 & 8 are executed in parallel to 5 & 6.
tLogCharacteristics Audit and error information from the MDM components are handled by
the tDie and tLogCharacteristics component.
tDie is used where a tCatch is used in the component logging message.
The messages are output to physical log files.
8 Log file Folder containing the physical log file.
9 tStewardshipTaskOutput This component will create a task for every run in DSC so that Data
steward can authorize/rollback changes in RDM.
5 MI REPORTING
MI will be directly querying RDM database to fetch reference data. MI work stream will be responsible to pull
reference data whenever they need.
6.1 Audit
The MDM out of the box audit capability will capture creates and updates to the data in the RDM database. The
journal is available through the Talend GUI’s to review.
Additional audit logging of messages passing through the RDM component architecture is required for reporting
and tracking purposes. This will be captured in a customized audit table, created in the MDM database, and
physical log files.
6.2 Errors
Error handling for RDM components is provided out of the box and will be part of the error and alerting
framework.
The handling of errors at message level will require a customized solution, the errors will be captured in error
tables, physical log files and an alert sent through email.
7 SECURITY
1 Public All publically available information available either externally Public displayed information such
or internally. (Always known as Public) as Support details, Web Content
2 Confidential Data used in normal business processes. The majority of an Customer and commercially
organization’s data would be included in this sensitive data stored within the
classification. Access to this data will require user Galaxy platform
authentication. (Previously and also known as Company
internal use only)
3 Highly Information, whose disclosure could seriously and adversely Financial data delegated to a
Confidential impact RSA. This would be a small subset of an Payment Provider
organization’s data and would include data and transactions
that fall under local regulatory guidelines, including
information security-related legislation.
8 AVAILABILITY
The backup and recovery requirements and approach (aligned to the RSA standards) are being managed by the
Galaxy Operations Architecture team.
At time of writing this design, the backup and recovery solution for the Galaxy estate has not been issued.
However the RDM solution has provided the following guidance.
Item Guidance
Availability RDM to be available 24 x 7 x 365 (other than at agreed maintenance times and
Requirements subject to unplanned outages) so that it is able to support MI reporting.
NFR’s are still pending sign off at time of writing this requirements, however it is
anticipated that 99.8% is the availability including unplanned outages. This excludes
maintenance period.
For maintenance period, where possible this will be done while system is available
to avoid breaking contractual agreements. But where required, to be agreed before
it needs to be taken down.
Round-Robin load balancing will be implemented (part of the Talend software
install).
An Active-Active setup with a Main and DR centre to provide fail over support to
mitigate against component/ server failures.
Enforce high availability by introducing redundancy and load balancing across
multiple servers (2x TAC servers, 2x MDM servers, 3x Execution servers for each
Active centre).
Backup & Recovery Weekly full backup (NFR05A.001)
Requirements Daily differential backups (due to frequency of the data updates and to ensure if
there is any data (user input or physical) corruption, we can roll back to a recovery
point in time - no more than 24 hours as per NFR05A.004).
Scheduled backups for checkpoint style recovery during migration end to end
processing.
Additional considerations for ETL parallel environment.
Recovery Backup process to be designed in such a way as to allow point in time recovery and
the ability to synchronize data in line with all other Galaxy MI components (The roll
forward from the weekly backup can recover data to any point up to the final
incremental backup that was taken).
The recovery Solution (which will be part of a separate design) to consider both
Operational and Disaster Recovery scenarios as the target recovery methodology
may be different.
Recovery processing will need to align services with other application servers in the
Load Balanced group.
Recovery processing will need to align services with other application servers in the
parallel processing environment.
MTA, Renewal,
Lapse,
* Cancellation codes
e.g.
* Partner trading
periods
* Referrer ID
As a Data Office * e.g. Partner agent 3.4.1 Met RDM defines capability to
Steward, I want details create/update reference data for
to To have the Partner/MI through a customized
ability for RSA Screen. The file per category has to be
Data Stewards to uploaded one by one.
manage partner
reference data in
RDM via a bulk
import process,
So that any
1810 reference data
Story
tables sourced
form partners
can be
automatically
uploaded into
MDMeither as
part of a
scheduled
workflow or
once approved
by data stewards
As a Data Office * Mass data update 3.5.2 Partially This will be a manual process. Every
Steward, I want * Batch upload met time reference data is created/updated
to Update (daily) in RDM, nominated data steward will
existing inform all impacted downstream
reference data systems including BizTalk giving them
on the fly & giving them specific time window to
build mapping make the changes. Once the changes
relationships to are done and confirmed by all systems,
the relevant data steward will authorize the change
enterprise notifying systems to make that change
systems, So that effective.
1811
Story when reference
data changes the
updates can be
made at any
time without the
need for a
systems release,
so that the new
table can be
scheduled to
take effect from
a specified date
As a Data Office * It is assumed any Met The Data steward will be given option
Steward, I want subscriber to new of choosing Impacted systems while
to Create new reference categories 3.4.1 & uploading the input file. Only the
reference data will need to be 3.5.1 chosen systems by Steward will be
categories configured to notified about the change.
where there is receive the
no downstream reference data
impact, So that
new reference
Story 1812 data tables can
be created by a
data stewards in
circumstances
where there’s no
inter-
dependency
with
downstream
systems
As a Data Office Met The data model stores canonical value
Steward, I want for each reference data which can be
to capture 3.2 & 3.5.2 used to find inter dependencies
canonical between systems to impact the
reference data change. Data model specs mentions
and any about the column supporting this.
mapping
associated to the
relevant
Story 1158.1813
enterprise
systems, So that
when creating a
changes request,
I can clearly
articulate inter-
dependencies on
downstream
systems
As a Data Office 3.2 Met Data model supports multi hierarchy
Steward, I want reference data. The entity reference
to to have the values can store n- level hierarchies.
ability to build
relationships
and hierarchies
1814
Story on reference
data , So that I
can provide
multi-
dimensional
insight
activity, So that I
can track
changes in
reference data
and can report
on transactions
based the
reference data
effective at given
point in time
As a Data Office 3.2 Met We are maintaining versioning with
Steward, I want effective start and end dates to meet
to set reference this requirement.
data to be
effective from a
given start & end
1816 date, So that I
Story
can report on
transactions
based the
reference data
effective at given
point in time
downstream
systems avoiding
the need for
multiple
updates.
promised to the
customer is
honored
As a MDM Tech Fixed Depth 3.2 Met Data model supports fixed depth
User, I want to Hierarchy hierarchy.
The ability to
support a data
hierarchy with a
1762 fixed number of
Story
levels, So that
RSA can define
filtering criteria
at any level of a
hierarchy
nodes to/from
multiple parents
As a MDM Tech Single Parent 3.2 Data Met Data model supports single parent
User, I want to Hierarchy Model hierarchy.
Single Parent
Hierarchy, So
1765 that I have the
Story
ability to support
a data hierarchy
with exactly one
parent for any
given node.