Search code examples
databasedata-warehousebusiness-intelligencebusiness-rules

360° Customer Data View


I'm working towards achieving a 360° Customer data view , I want to bring together data from multiple systems across the business. I expect this is a broad question but is the only way to achieve this by creating a data warehouse?

Thank you


Solution

  • The answer is: It depends.

    The absolute minimum requirement is that you are able to link customers from the different source systems.

    In theory, you could create a bridging table with the natural keys (unique IDs) from the various source systems as columns, fill it using some kind of matching algorithm and let the reporting platform deal with the joining of the data.

    In practice, you will run into a world of pain on all but the simplest of systems when you try this:

    • Endless data mismatches ruin faith in the accuracy, since there are no common definitions between source systems.
    • Report generation performance is unacceptable with many, many joins
    • Production systems may be impacted by report queries
    • Changes to source systems break reporting in unexpected ways

    Setting up a datamart or data warehouse is meant to solve all of those problems. You pull in production data once rather than for every report, clean and validate the contents, transform it into reporting-optimized forms and present it to end users in a consistent way.

    If your 360° view is focused on clearly defined transactions/interactions, especially if they are interrelated (i.e. support tickets relating to products purchased recently), then investing the time to analyze all the incoming data and unify the meanings of similar data from different sources is likely the only sensible option. Writing all those rules in the ETL layer of a Data Warehouse design makes it both more performant and easier to maintain than embedding the logic in each report separately.

    If you instead have a bunch of data scientists running their own analyses that don't need to see numbers matching to the last digit every single time, you could probably get away with dumping semi-structured data from each source system together into a NoSQL DB or similar storage with only the customer unique ID to unite them.