1. Lack of Business Objectives
1. Lack of Business Objectives
Many organizations fall into the trap of collecting and analyzing data purely for the sake of doing it. The problem is without a clear business objective, the data is unlikely to yield any clear benefits or useful business intelligence. Before even purchasing a big data resource, organizations should consider where the gap in their current data and analytics is and how a big data platform can fill that gap. Big data should then be used to help answer specific questions or contribute to new initiatives. If a data set is failing to fulfill a specific need, it should be abandoned for a set of real value to the company.
2. Insufficient Data Quality
Big data is complex and consists of multi-structured data that can be inconsistent and can lead to erroneous results if business leaders aren’t paying attention to the quality of the data they are using. For example, in the healthcare industry a lot abbreviations are used that can overlap and misconvey the meaning of a doctor’s report.
Unless the multiple meanings of these abbreviations are accounted for, the statistics on the number of heart attacks could be completely inaccurate because another condition with the abbreviation “HA” is being included in the report. In addition, ensuring quality is difficult because the data is presented in many different and ambiguous formats. Consider the difference between a comment on a user forum and a tweet on Twitter. One uses complete sentences while the other uses a handle, hashtags and short, abbreviated phrases. Failure to recognize this difference when setting up an analysis could result in skewed results.
3. Lack of Expertise
As big data platforms are relatively new, most organizations don’t have trained teams in place to use the platforms successfully. Relying on the IT department to drive big data insights will not accomplish business objectives. Rather, whichever teams that are needing access to data, such as marketing or sales, should be trained how to use the data. The IT department may need training too on how to facilitate and integrate the platform, and if someone is not designated to oversee big data operations, someone should be brought in who can.
4. Using Irrelevant Data
Just because big data allows you to use huge data sets does not mean you should include all of your data in an analysis. While comments on social media may be helpful in an analysis of consumer sentiment, it will not be helpful in improving internal efficiency standards. Before using a data set determine what the data set is relevant for and only use if for that purpose. Don’t waste time and money using irrelevant data that may skew the results you are seeking.
5. Not Accounting for Human Error
Data has always had elements of human error, and big data is no different. Typos and other inaccuracies, such as confirmation bias have the potential to skew a data set and upset results. Even government intelligence agencies have run into this problem with suspected terrorists’ names being spelled differently making it difficult to keep track of them.
While a perfectly clean data set may be a sign of fraud, businesses should put guidelines in place to eliminate error and account for it in its analyses. Confirmation bias, in particular, should be protected against, as managers tend to dismiss data that doesn’t fit their own assumptions and cherry pick data that supports their ideas.
As big data platforms like the Hadoop become more prevalent it will become easier for business leaders to use big data more effectively. Ultimately though, deriving value from big data will depend on business leaders learning how to apply it to their business.