Table of Content

INTRODUCTION

Over the past few years, leading companies have implemented CRM technologies to gain a better understanding of the needs of their customers, to improve customer satisfaction and increase profitability. An effective CRM strategy is an important first step in achieving that goal. To gain real competitive advantage, winning organizations must incorporate a Unified Customer View – a single, unified view of each customer – accessible across every facet of your business.

A complete integration solution must address several critical issues: customer data rationalization, an open data model to integrate customer information, and a set of services to manage and share this integrated information.

Many companies have implemented best-of-breed applications for a variety of business purposes including Call Center management, Sales Force Automation, Order Management and Financial Accounting. Although optimizing local functionality, this has often resulted in scattered distribution of vital customer information across multiple systems. The complexity of developing a unified view of one's customers is compounded when the growing number of touch points is factored into the equation. Customers may contact your company via email, website and many continue to opt for phone interaction with a customer service representative.

Xoriant Customer Integrator™ (XCI) provides all the necessary tools, data models and best practices to rapidly deploy your customer data integration strategy. Customer Integrator enables real-time integration of customer data from incongruent data sources within your enterprise to achieve a unified customer view.

Xoriant Customer Integrator is an innovative, standards-based product designed to rapidly implement your Customer Data Integration strategy. At the heart of Xoriant CI is the Customer Directory - a unique, powerful data model to track customer identities across all your systems. Xoriant CI comes with all the tools necessary to rapidly build your Customer Directory, establish global attributes (like a global customer identifier), dynamically generate views of customers from diverse systems and share this customer information through a web-services architecture.

TECHNICAL CONCEPTS

Unified Data Model

Typically, any large corporation has multiple enterprise systems. Each of the enterprise systems has its own data model. This may be in the form of a database schema or an entity model. The goal of XCI is to provide a single global data model, which may be a union or a subset of the union of all the data models. The definition of this single data model is driven by business needs. The resulting value is that the enterprise does not have to choose the data model of a single silo as a master. Rather than promote a “master-slave” data integrity strategy, XCI allows the implementation of a true “federative” strategy in which independent models can coexist and still contribute to a unified enterprise view of key entities such as Customer.

Using this single (unified) data model, an enterprise can:

  • Feed data to applications from various enterprise systems without the application being aware of the enterprise systems.

  • Update information in the enterprise systems transparently.

  • Keep data in all enterprise systems in sync. For example, if a customer’s address changes in one system, automatically change it in other enterprise systems.

  • Add new attributes to the unified data model – which need not map to any attribute in any of the existing enterprise systems.

The first step towards achieving these goals is to uniquely identify entities in various enterprise systems. A central directory that holds references to the same entity in various enterprise systems provides the link between the same entities in various enterprise systems. For example, the same customer may be in represented differently in different enterprise systems. Xoriant’s Customer Integrator holds Customer IDs from various enterprise systems, allowing people and applications to locate the fragmented customer representations in various enterprise systems and build a consolidated view of that customer as understood across the whole enterprise.

Entity Models & Maps

XCI is designed to be agnostic regarding a data source system’s data model. To achieve this goal, we define an XML-based input to express the data model, which we call Entity Model. The Entity Model is not necessarily tied to RDBMS even though it models the notion of primary and foreign key relationships. The entity model can also be used to represent objects whose definitions are not fixed (that is, instead of defining a Java Class to represent an entity).

To move or transform data from one Entity Model to another, we provide mapping between models, which we call Entity Model Map. This tool is an XML-based input to express the mapping between entities in one model to entities in another at the attribute level.

The map specifies various types of relationships between data source entity attributes and target entity attributes, viz. 1-to-1, many-to-1, 1-to-many, etc. In the 1-to-1 relationships, it says how an attribute in one data source maps to an attribute of the target Entity. Along with such mapping, it also enables the ability to perform some translation and transformations. For example, temperature in Centigrade can be translated to Fahrenheit, a single attribute can be split and mapped into multiple attributes (1-to-many relationship) or several attributes can be combined to form a single attribute (many-to-1 relationship).

XCI employs well-defined XML schemas for both Entity Model and Entity Model Map. The Entity Model contains Entity definitions and the Entity contains Attribute definitions. The schema allows annotating the objects (Entity Model, Entity or Attribute) with properties. This makes the schema extensible, each client application can interpret the properties in its context.

COMPONENTS

Xoriant Customer Integrator uses a lightweight directory architecture to ensure scalability and
performance, to support multi-system entity record cross-indexing, and to avoid the bottlenecks of traditional data warehouse-oriented integration. With this architecture, an institution can store summary data in the Entity Integrator database for touch point responsiveness while detailed transaction data remains up-to-date in source applications, ready for access whenever and wherever needed.

XCI comprises of the following component modules and functionality:

  • Extractor

  • Cleansing and Matching Engine

  • Customer Directory – Builder and Server

  • Scheduler

  • Synchronization

Extractor

Extractor is used for extracting data from different data sources. It is used for initial migration of data from data sources to build the Customer Directory and also to periodically process any additions/updates to the data sources.

It loads the minimum matching data for the “Customer” as a primary entity being unified from the various data sources. This data goes through the Cleansing and Matching process before a directory is created or updated.

XCI uses Extractor to get customer data from various source systems. This process of extracting data from different systems is called Extraction.

Cleansing and Matching Engine

The Cleansing and Matching Engine (CME) implements a comprehensive cleansing and matching process to standardize your customer data.

Why Cleansing and Matching?

Data Cleansing and Matching is required to ensure high data quality. It is required to ensure that the information in a given record is unique, standardized, logical and consistently formatted.

The Cleansing and Matching Process

The Cleansing and Matching process involves the following stages:

  • Data Qualification

  • Cleansing

  • Matching

  • Manual Matching

Data Qualification

This stage validates the source data for cleansing and matching. For example, using the rule called Required Column Rule. You can check for the presence of information in specified attributes of the data. The records that are cleared by this process form the ‘qualified’ data. These records, can then be cleansed to bring about data consistency.

Cleansing

The purpose of the data cleansing process is to capture data from all source systems and reconcile definition differences so the unified customer model in XCI can present a consistent view of a unique customer to all users within the organization.

Data cleansing straightens out any inconsistencies in the data itself. Customer names are a classic example of data that can be different in each source system (e.g., "ABC Inc." or "ABC Incorporated" or “AB#C Incorporated”).

Matching

The data matching process helps in reducing the duplication of data in the master system when data is received from various systems within an enterprise.

CME compares the confidence levels and classifies the records in the source data into three types:

Duplicate records: The confidence level that the source data matches an existing record is greater than the specified maximum confidence level.

New records: The confidence level that the record is unique is less than the specified minimum confidence level.

Ambiguous or Uncertain records: The confidence level of source data record is in between the specified minimum and maximum confidence levels. CME inserts these records to the Manual Matching database from where they can manually compared to decide their level of matching.

Scenario 1 – Real-Time Usage

In the real-time usage of the CME, an external application calls the CME through a Java/XML interface. The input data goes through the Qualification and Cleansing stages. The data is then matched with the data from the system of records by CME and CME returns the status of the match through the Java/XML interface. The external application can then take action based on the inputs from CME.

Scenario 2 – Batch Mode Usage

In the batch mode usage of the CME, the data is fed to the CME through a Java/XML interface. The input data goes through the Qualification and Cleansing stages. The data is then matched with the data from the system of records by CME. If the input data record is of type ‘New’, it is added to the ‘Staging Area’ and a Data Loading Process loads it into the Staging database. Once the record has been added in the Staging the next record is processed by the CME. The data record of type “Uncertain” goes to the Manual Match process.

Rules Engine

The Rules Engine Module of XCI enables you to create and manage rules, rule sets and specify the attributes on which the rules have to be applied.

Defining Attributes: Define / Configure all the entity attributes that are to be applied in the rules using the Rules Engine tab / Attribute option.

Defining Rules: Define / Configure all the entity attributes that are to be applied in the rules using the Rules Engine tab / Attribute option.

Defining Rule Sets: Define / Configure all the entity attributes that are to be applied in the rules using the Rules Engine tab / Attribute option.

Manual Matching

Manual Matching is the stage of Cleansing and Matching where you match the Uncertain records with the records in the reference database. Uncertain records are the records classified as ambiguous during the matching stage.

Customer Directory

Customer Directory is a lightweight directory, which uniquely identifies a customer in all the data sources by building a cross-reference of customer IDs across multiple applications. It maintains a unique identified called Global ID for every new instance of a customer.

Customer Directory is a lightweight directory, which uniquely identifies a customer in all the data sources by building a cross-reference of customer IDs across multiple applications. It maintains a unique identified called Global ID for every new instance of a customer.

For example, - Global ID 1001 may mean Customer ID 2001 in Siebel (CRM) and Customer ID 3001 in Oracle Apps (ERP). This directory also maintains the matching attributes required for identifying the customer uniquely. The matching attributes help to check if the customer already exists in the directory.

Customer Directory Building Process

This process involves multiple steps such as extracting data from source systems, cleansing and matching this data etc. This can be done either in near-real time mode or in batch mode.

As mentioned above in the Introduction section of this chapter, customer directory maintains the cross references of the customer Ids from enterprise wide systems and applications.

It has its own user interface to manage various administrative as well as directory specific operations such as editing global name.

Need for Customer Directory

Once the directory is built, it can be used for various purposes such as

  • Providing a unified view of customers (360-degree view)

  • Providing a Customer Master with user defined attributes

  • Streamlining the customer creation process

  • Improving the quality of customer information

  • Empowering other applications such as Reporting and Customer Analytics with centralized customer data

  • Helping automate business processes based on the data in this directory

Customer Directory "Identity" Server

The XCI Customer Directory is accessed through an Identity Server that provides a number of services for storing, reading and updating Directory information. The following functions can performed using the Customer Directory:

  • View and Search on customer data

  • Search by ID – Global ID and Business Application ID

  • Search by Name – Global Name and Business Application ID

  • List all the global IDs

  • Based on Global ID, view list of Business Applications

  • Based on the business application ID, view address and demographic details

  • Merge Customer data under one Global ID

  • Split Customer data into multiple Global ID

  • Resolve Conflicts

  • Configure UI properties

  • Generate pre-configured reports

Scheduler

XCI uses Scheduler to set up the data extraction or cleansing and matching processes. Using Scheduler, the processes can be set to run at any interval of time in the multiples of Minutes, Hours, Days, or Weeks.

Synchronization

Once the directory is built and loaded, it can be used to "push" clean and validated customer data to other applications. Synchronization is a way to send or get data from Customer Directory.This process currently supports interfaces to extract data from the Directory in batch mode. Future releases will have ability to drive this process based on events executing in different systems.

ARCHITECTURE

Xoriant Customer Integrator is architected as a modular set of standards-based components to ensure performance, reliability, scalability and adaptability. As illustrated below, this component-based architecture allows XCI to be easily integrated into any technical environment without requiring significant retrofitting of existing systems.

SUPPORTED PLATFORMS AND TECHNOLOGIES

Software

Application server

J2EE (with EJB 2.0) compliant application server (e.g. Web logic 6.0 SP2)

Database server

Oracle RDBMS version 8.1.7

Operating systems

Windows NT4.0, Windows 2000, Solaris 7.0 onwards

Development platform

Java JDK 1.3 with EJB 2.0

Web Server

WebLogic web server

Client Browser

Internet Explorer 5.0 and above, Netscape Communicator 4.5 and above

Hardware

Intel based PC/Server

800MHZ or more CPU

512MB RAM (Minimum requirement) 1GB (Recommended)

20 GB Hard drive, additional disk needed based on size of data

Sun Platform

400 MHZ or more Sparc CPU

512MB RAM (Minimum requirement) 1GB (Recommended)

20 GB Hard drive, additional disk needed based on size of data