Fix data before warehousing it

Many data warehouse operators have attempted to implement Master Data Management to improve data quality, but most have focused on mastering data after transactions occur. This approach does little to improve quality because data is "fixed" after the fact. The best way to improve data quality is to move the process upstream of the data warehouse and before transactions are executed.

One goal of MDM is to prevent bad data from entering the ecosystem at all by enforcing data quality at the edges of the IT ecosystem. For example, when creating a sales order, the transactional system should guarantee that customer data on the order is not duplicated, and that address data are correct and current. The same thing could happen for the product data, prices, discounts, and payments on the order.

The system should also enforce business rules and policies and guarantee that the transaction is correct. Master data provides an enterprise-wide purview of data, whereas transactional systems only have insight into the data they contain. Since each source does not contain the sum total of master or reference data required to guarantee enterprise-wide data quality, they only provide a partial solution. This is why MDM can play a critical role.

If MDM is implemented using a real-time architecture, based on the principles of service-oriented architecture (SOA), then a service that enforces the business rules and policies governing each kind of master data can be made available to all transactional systems.

For example, a service that manages the uniqueness and quality of customer data can be created to ensure that a customer is created only once and made available to any system of record whenever a transaction occurs. This satisfies the need to enforce data quality at the edges of the ecosystem. The same applies for any master data including data about people and organizations of interest, products and related catalog and pricing data, locations of customers, products, and company assets, calendars of events.

These services, which can be called Master Data Services (MDS), guarantee the integrity of the most critical data within every transactional and data capture system in the organization. Each MDS can be an autonomous, decoupled service that is individually scalable and managed to ensure the quality of the domain of master data it manages.

One of the prime services provided by an Enterprise Service Bus (ESB) should be a MDS that serves as the real-time "single version of the truth" for a specific kind of master data. This type of service contributes significant business quality improvements because it provides data to the data warehouse and to all IT systems that consume or refer to master data.

Sign up for ITworld's Daily newsletter
Follow ITworld on Twitter @IT_world

• Email
• Print
• Share
  • Slashdot
  • Digg
  • Stumble
  • Reddit
  • Newsvine