Make Data Human: Apply Deming’s Principles to Data Analytics

 It was a sunny spring day. I just received my coveted birthday present. A Sony Walkman. It was revolutionary. Small, lightweight, beautiful, it was music on the go. And it was the essence of Japanese ingenuity.

W. Edward Deming knew a few things about counting: He was the son of a chicken farmer in Iowa, USA. A trained mathematician, he worked as a census consultant to the post-war Japanese government.

While there, he was asked to hold a short seminar by the Japanese Union of Scientists and Engineers. He taught statistical process control and concepts of quality.

Deming called his system of thought “System of Profound Knowledge”. His message to Japan's chief executives: Improving quality will reduce expenses while increasing productivity and market share.

Many more seminars followed, with one of the attendants being Akio Morita, the cofounder of Sony. 

Industrialization of data analytics

Deming’s methods profoundly transformed the industrial processes in Japan. It’s time to apply these same concepts to data analytics.

74% of firms say they want to be “data-driven”, reports Forrester. Yet only 29% are actually successful at connecting analytics to action.

Rajeev Ronanki et al. of Deloitte Consulting pointed in a recent blog post to some of the reasons for this apparent contradiction. They outline:

“Advances in distributed data architecture, in-memory processing, machine learning, visualization, natural language processing, and cognitive analytics have unleashed powerful tools that can answer questions and identify valuable patterns and insights that would have seemed unimaginable only a few years ago.  Against this backdrop, it seems almost illogical that few companies are making the investments needed to harness data and analytics at scale. Where we should be seeing systemic capabilities, sustained programs, and focused innovation efforts, we see instead one-off studies, toe-in-the-water projects, and exploratory investments.”

Data Analytics Principles

It’s time to change and a good place to start are Deming’s methods. Deming advocated in his System of Profound Knowledge four key points:

• Appreciation of a system: understanding the overall processes
• Knowledge of variation: the range and causes of variation
• Theory of knowledge: the concepts explaining knowledge and the limits of what can be known.
• Knowledge of psychology: concepts of human nature.

Let’s apply these four points in turn to data analytics.

• Appreciation of the system: any analytics initiative should be setup with the goal of improving products or services. This may include suppliers, producers, and customers (or recipients) of your goods and services. Any analytics initiative must have to goal to provide novel, timely and actionable insights in context, relevant to specific production process.

• Knowledge of variation: Analytics today is correlation. Regardless of the level of sophistication any correlation has statistical sampling issues.

• Theory of knowledge: Deming railed against blindly asserting opinion as fact, out of convenience or ignorance. At the start of any analytics initiative a company lacks the frame of reference to validate and assess results. A good way is exchange results between industry partners and providers (we’re ready to share ours) to learn what is necessary to improve the situation. Learning needs to be continual and organization-wide.

• Knowledge of psychology: Deming understood the fundamental truth that people are different. Indeed one can create the best analytics system, know all about variation and still have a failing analytics initiative. The key is to understand people, and particularly what motivates them. The transformational effects of analytics are profound. The key to is make people not just part of such a journey but address intrinsic needs, including taking pride in workmanship and working with others to achieve common goals.

Example: One of our customers is rolling out our Service Insights solution. The key goal: optimize their call center response times by up to 30% (in fact deploying the pilot results across the entire call center). As part of the initial project setup we involved the call center agents in the actual design of the solution.

The effect: the agents were driving the project. It was no longer a management imposed efficiency initiative but a team effort to improve their workplace The team made use of data to transform their organization. In a way they made data human.