Master Data Management in Financial sector


One of the problems with large banks is that they are monsters created by mergers and acquisitions.

The initial plan would be to merge the software stacks in a phased manner.
  • Identify systems that provide common functionality.
  • Identify costs associated with maintaining and upgrading the systems and software.
  • Select a strategic platform.
  • Start your project to migrate all applications onto the strategic platform.
  • Start Phase 1
  • Get lost in business process translation.
Most of the time it is too hard to merge two software stacks and come up with a single unified strategic platform. Higher Management would see this as a futile exercise and will take a decision to maintain the two systems and merge them in the future to save costs in the short term.

Now, from a customer perspective you are a single bank, but the from the bank perspective you are a different customer in each of their systems.

To simplify things, let us assume a bank which is standalone and does not have the baggage of mergers and acquisitions. Until unless the bank is established in the 21st century the core software landscape will be a mix of modern and legacy applications.

Most of the time, they will be customised 3rd party products. To make matters worse, the original product will be out of support and with zero documentation. In this kind of setting, it would be a humongous task to integrate all the existing systems to provide a unified view of the customer data.

A typical software landscape would be Core Systems:
Infrastructure  Software Persistence
 Mainframes, AS400  Cobol, CICS, IMS  DB2, IMS
 Linux  Java, J2EE, EJB  Db2, Oracle, SQL Server
 Windows  Seibel  Oracle
 Linux  SAP  Oracle

Customer and staff-facing applications:
Infrastructure  Software Persistence
 Application Servers on *nix Java, Spring  DB2, Oracle
 Windows .Net   SQL Server
 Browser  Angular, ReactJS
Mainframe  CICS screens  IMD, DB2

To marry all these disparate systems, the data and integration architects will most probably come up with an SOA-based application and data integration architecture. Service Oriented Architecture works well as an application integration architecture.

For example, the mainframes systems can be integrated with Java front-end system by using an Enterprise service bus or a Webservice exposed from CICS. But the data is still restricted to the interface the core backend system exposes to the frontend.

Over the course of time, all new requirements will be catered in the front end layer rather than making changes to the core systems. The reason can be cost, flexibility and human resource. This leads to data fragmentation and the concept of "single source of truth" no longer applies.

To counter this the organisation will create a centralised datawarehouse to house all organisation data. This would help in consolidating data at a single place for consumption by the organisation, but most frequently the data is not real time. The aggregated data in Data warehouses would be flat, complicated and without any logical structure. This data would then be used only for offline reporting or data analysis.

Even after such complex integrations and data structures, the organisation cannot make any real times decisions on the data.

To solve this problem there are many products in the market which can help organisations better use their business data to achieve expected business outcomes. Some of them are
  1. A Hadoop cluster to house enterprise business data.
  2. A real-time global commit log for stream processing.
  3. A graph database/structure to house customer data along with entity relationships.
  4. A NoSQL database like SAP HANA.
A typical implementation of a Master Data Management system can be as below
  • All enterprise data will go through a single communication highway, the Global Commit Log. In this example, I will use Apache Kafka as a distributed commit log.
  • All the systems first write any of the data updates into the commit log before clients to the log read the data and place them into their individual data stores.
  • Datawarehouse and Hadoop systems stream data from Apache Kafka at their own pace.
  • All business entities along with their relationships are moved to a global Graph Database. Example: Neo4j.
  • Any updates to the entity will be reflected in the Graph on a near real-time basis.
  • This Graph will be used for any read queries related to organisation data.
  • All updates/additions still flow through Kafka.
  • Graph queries can be cached and any updates can be propagated by a push mechanism.
MDM.jpg    

Comments

Post a Comment

Popular posts from this blog

Understanding SNOMED-CT data model: Basics

Image
Ever wonder why specifications from a standard body are complex? Have a look at Java Specifications or try understanding a new declarative language, they are mired in complex functionality and assumptions. A couple of years ago, I came across SNOMED-CT and found that not all specifications are made equal. Being an IT professional, I expected such kind of work from an IT major, but surprisingly it came from the medical community. SNOMED-CT is a specification for identifying, defining and describing medical terms. This should have been one of the complex specification in the know IT world but is one of the simplest. The authors of this specification understood the underlying structure of the data should closely represent the domain model, which is the medical terminology context. The following is the basis for the specification(I will use terminology and specification interchangeably in this blog): All components in the terminology are represented by Concepts. Any re...
Read more

Why the role of IT in Financial Institutes is more important than ever

Image
Most of today financial institutes see IT expenditure as an expense which eats into their profit margins. But, in todays world where you can order virtually anything on the Internet.  Interestingly , you can buy the Moon . But, try buying an insurance or apply for a loan and you would spend anywhere between 1 hour to days to get the policy or loan approved. Why does this happen? Why organisations tend to see IT as an expense rather than as an Investment? From a finances perspective, IT expense does not convert to revenue. Something similar to using fuel in a Taxi. For a taxi driver the taxi is an investment and the fuel consumed is an expense. Even though his taxi does not run without fuel, it is always seen as an expense. This would mean, sometimes it might seem to be a good idea to use subpar fuel to improve profitability. Businesses cannot survive without IT it is the life blood of modern organisations.But from a finances perspective it looks like an never ending ...
Read more