The Post-it Notes of the Big Data World



This past weekend I decided to tackle the mess in my basement.  In my last, I compared the modern data lake to my “thing lake” in the basement – a random collection of stuff that may, or may not, be useful.  So my big job last week was organizing my basement.

The Post-it Notes of the Big Data World

I went through every item and placed a post-it note on it – the note described what it was (exercise equipment, baby clothes, etc.), who bought it, who used it.  I even got different colors for different classifications of things.  It took hours.  But was my basement any cleaner?

Was anything more useful?  No.  Well at least, not yet.  It was a useful step towards actual organization – throwing things away, re-organizing them into sections by their classification, etc.  But there was a lot of work ahead of me to make my “thing lake” really organized and useful.

Managed data lake solutions

Managed data lake solutions are the post-it notes of the big data world.  They use meta data to classify the data within the lake.  Some visualization tools do the same.  But then the work is shifted back to the data scientist to join the data sets, determine what is useful, and somehow merge it into a useful larger concept, such as a customer record.  Those solutions are a necessary step to organize the lake, but they are only the first step.


If you are like most companies, the number one data domain you are interested in within the lake is customer.  And therefore you need solution that not only classifies raw data, but actually organizes it into a Customer 360Customer Intelligence Management Systems do exactly that.  They synthesize data into a Customer 360.  They use machine learning analytics to infer intelligent attributes for the customer record.  They evaluate confidence scores for all aspects of the Customer 360.  They visualize customer data and present perspectives to different user audiences.  And they maintain the Customer 360 for operational and analytical use.


Customer Intelligence Management Systems benefit from managed data lake or meta data tools classifying everything in the lake, as it aids in the synthesis process.  But if you really want to transform and use the customer data within your lake, you’ll want to move beyond managed lake tools and towards a Customer Intelligence Management system.  To see the difference and how organizations use CIM, check out this demo.

All Credits to David Corrigan