In a blog post titled Crazy BI, Jorgen Heizenburg, Principal Technology Officer for Business Intelligence at Capgemini Netherlands discusses the challenge of using external data to improve the performance of BI, identifying 3 main problems: [1] too much data, [2] data too late and [3] poor data quality. He concludes by arguing that companies need to structure and combine internal and external data in order to gain any competitive value.
I agree with Jorgen on this and the general thrust of his post, but I had to respond to his suggestion that we should “forget about data quality”. Here’s my response …
Crazy BI
Interesting post Jorgen, but I must pick you up on the issue of data quality.
If you are saying that you shouldn’t wait for your data to be perfect before using it in BI, then I agree; but to completely ignore the quality of the information you’re using to inform your decisions would be like playing roulette – Russian style. I’d also suggest that having too much data or data that is out of date are very much data quality issues.
No, you don’t need perfect data for BI, but you do need data that is fit for purpose and you therefore to be able to define what good data looks like and how you are going to measure (and if necessary) improve its quality.
These challenges also apply to external data, which all too often, imho, people see as a silver bullet. You need to understand the provenance of that information – where did it come from and when was it collected? And unless your’s and the external share a common key, you’re going to have to use some intelligent processing to integrate it in a way that will deliver value to the business.
As an industry we like to label things, but I see data management, data integration, data quality and data governance as different expressions of a desire to do the same thing – the creation, management and maintenance of data that is fit purpose for the business – all the purposes of the business.
Delivering on that requires processes, tools and technology that support all aspects of all the aforementioned list, but most importantly of all, it needs the recognition of the value of data to the business. For more of my views on this subject, visit my blog on www.datanomic.com.
I truly do believe that Business Intelligence without data quality would be Crazy BI!
