The Problem with Traditional BI Software
A white elephant is a possession which its owner cannot dispose of and whose cost, particularly that of maintenance, is out of proportion to its usefulness
The Problem with Traditional BI Software
A white elephant is a possession which its owner cannot dispose of and whose cost, particularly that of maintenance, is out of proportion to its usefulness
A mere 10-15 years ago, Business Intelligence software was still considered the sole dominion of large, fortune-500 scale companies. These were the types of organizations which typically had a lot of data, and also the ones that had the vast computational resources which were then required to process this data and translate it into actionable insights.
The BI market was dominated at the time by traditional enterprise tools: heavy-weight, large scale software, requiring months of dedicated IT work to set up and a continued effort to maintain and operate – not to mention the multi-million dollar investment typically required, both up-front in hardware and implementation fees, and in continuing operational costs.
I will claim that this model of Business Intelligence – which for the sake of this article we will refer to as traditional enterprise BI – has become too big for its own good, and has become a “white elephant” for many organizations; and that the marketplace is moving towards an alternative design: namely, Agile Business Intelligence.
The Source of the Problem: Outdated Technology
To understand the problem with traditional enterprise BI, one has to look at the underlying technology that powers it, and why this technology is simply not a good fit for the needs of modern organizations.
30 seconds about OLAP
Traditional enterprise BI tools typically rely on Online Analytical Processing (OLAP) to join different data sources into a single source of truth called a Data Warehouse. While these solutions are designed for scalability, they require a significant investment of time, effort and money as well as a staff with the technical know-how to operate them.
OLAP technology relies on pre-aggregating results to pre-defined queries. Creating new data imports is lengthy and requires extensive IT support. Queries are also complex and lengthy to set up and require professional knowledge (usually coming from the IT department).
In an OLAP platform, calculations are performed when the system is not being utilized by end-users, resulting in fast answers to pre-defined queries, but very limited support for ad-hoc queries. New questions coming from the business side, or ones that involve taking new data into account, could take weeks or more to answer, depending on the company’s IT and hardware resources.
With this in mind, it’s clear that an OLAP based-system is most useful in organizations that:
Have a clear cut list of predefined data sources
Possess a large amount of resources to invest in their BI solution
Don’t mind waiting for prolonged periods to receive answers to new business questions
Why It’s Just Not Working
These inherent limitations of OLAP technology present a true challenge for traditional enterprise BI. The three characteristics we have detailed above are simply not true for more and more companies who are currently in the market for business analytics software. Here’s why:
More relevant data sources
With the emergence of Big Data, companies can no longer rely on their existing database to remain more-or-less static in the future. New sources of data are constantly appearing, with data collection and storage becoming cheaper, more sophisticated and more automated. In this state of affairs, the ability to quickly create new data imports and immediately be able to take them into account in decision-making process is crucial.
Smaller companies and departments with limited resources
Today, it’s not only the fortune-500s who are looking to make business intelligence an integral part of their commercial strategy. Much smaller companies are also hoping to jump on the “data bandwagon” and utilize data to make more informed decisions and improve the efficiency of various processes within the organization. These companies often lack the resources or willingness to enter a long-term, millions of dollar investment which traditional enterprise tools require. Even in companies which can afford these traditional tools, many departments are finding them unsuitable for their needs, and find it wasteful to implement these huge, heavyweight platforms for the analyses they wish to perform.
Actionable Insights require fast answers
The need to wait days if not weeks for answers to new business questions is extremely problematic, and severely limits end users’ possibilities to perform their own analyses and data discovery. It forces business users to view their data through the very narrow windows which were pre-built for them by their IT department, leaving much to be desired in terms of self-exploration within the data and discovering hidden insights and connections.
In this state of affairs, it’s clear why traditional BI software has become somewhat of a white elephant: It’s been there for ages, it requires immense resources to keep up, but nobody is really sure if it’s providing true value anymore.
A Challenger Appears: Agile BI
Considering all of the above, why is the Business Intelligence and Big Data market hotter than ever? Is it a bubble waiting to burst?
I will argue that this is not the case — that the promise the field is showing is due to the growing prevalence of Agile BI software. These are new software tools that look beyond the limitations imposed by OLAP, and seek to find technological alternatives that will give end users similar performance and capabilities — but without the restrictive costs and overall burden of setting up a traditional BI solution.
