The Tyranny of Bad Data
We’ve all experienced the frustration and pain associated with bad data – either we’re aware that the information obtained from IT systems is inaccurate or incomplete (and the non-value added work that comes with it) or unaware that the information is based on bad data and the cascading impact of taking the wrong course of action due to misinformation. It is incredible that so much is invested on enterprise technology solutions, as little attention is devoted to ensuring high quality data is the sole source of system information. Only a handful of companies have discovered how to exploit the power of lean IT to shorten time to value development cycles, while assuring data integrity.
The ultimate purpose of information and technology is to enable people to perform great work as effectively and efficiently as possible. From a lean IT perspective, we want to leverage technology to empower people to do excellent work with the least amount of required effort. Technology has the capability to gather, store, organize, manipulate, manage, calculate, analyze, summarize, format, and report limitless amounts of data in order to create actionable information. Technology that efficiently delivers bad information only serves to enable waste, delays, and poor results. For our purposes, information needs to possess the following attributes to be deemed actionable: accurate, timely, complete, and accessible.
For a practical example of Lean IT and data management, see the webinar Lean IT: Driving SAP Continual Process Improvement.
When bad data happens to good people
Donald Rumsfeld, former US Secretary of Defense infamously said: “You go to war with the army you have, not the army you might want or wish to have at a later time.” In the same way, we do business with the data quality we have, not the data quality we might want! But what happens when highly effective technology processes inaccurate, incomplete, and out-of-date data; when bad data happens to good people?
Scenario #1 – We know we don’t know…
If people recognize that the information they are receiving is not actionable, they are forced to choose from damaging alternatives like adjusting their course of action based on years of experience, assumptions, and perceived understanding. Some develop rules of thumb based on personal knowledge, while others devise creative workarounds to obtain the information they require when system information is suspect and unreliable.
Unfortunately none of these countermeasures confronts the root cause of the problem, nor guarantees a timely and accurate business outcome. Undocumented workarounds and tribal knowledge of what to do when the system delivers bad information may work in one instance and fail in another, and all of these actions are forms of guessing that are impossible to scale and sustain.
Scenario #2 – We don’t know we don’t know…
When people rely on information from IT systems, assuming accuracy, timeliness, and completeness, and that information is actually compromised, things get much worse. Bad data generates bad information, prompting people to make misinformed decisions, mistakes, oversights, and the creation of more bad data! The compounding impact of bad data and inaction-able information is a frustrating, downward cycle of errors, corrections, rework, and delays that force people to resort to heroic efforts to deliver mediocre results. Customers instantly notice a lack of service, timeliness, and quality. As employees become more aware of data problems, they begin to lose trust in the system and resort to the workarounds described in scenario #1, which may feel better and attain some results, but do not materially improve the situation. In fact, the more exceptions and workarounds to the way work is conducted, the more variability the customer experiences in service levels, quality, and delivery time!
In my next post, we’ll explore How Lean IT addresses the issue of bad data at a level that creates measurable, sustainable change for the better.