There are many challenges to incorporating big data into analytic processes – technical, skill set, and cultural challenges for example. But you must also consider the challenge of thinking differently about the analytics you need to do and how you will go about doing them.
There are many challenges to incorporating big data into analytic processes – technical, skill set, and cultural challenges for example. But you must also consider the challenge of thinking differently about the analytics you need to do and how you will go about doing them.
You can’t just focus on how big data can improve existing processes or methods for solving the same old problems. You also can’t just focus on the totally new problems big data can help you address. You must look for ways that big data can enable an entirely new approach to addressing an existing problem. This last point is what I will focus on here.
How Big Data Can Change Longstanding Standards
In the pharmaceutical industry, the longstanding method of assessing drugs is the use of clinical trials. Clinical trials are a gold standard for sound analytics. In a clinical trial, true treatment and control groups are created in a setting where participating patients and doctors don’t know what drug a participant is getting. As many external factors as possible are accounted for up front so that the impact of the drug can be assessed accurately. The methodology is rock solid and it is the only approach allowed in many cases to prove the safety and effectiveness of a drug.
Clinical trials have drawbacks, however. They are insanely expensive, for one thing. They also typically end with fairly small sample sizes. After years of effort and hundreds of millions of dollars, a study can be lucky to end with a sample of just two or three thousand patients. While the studies are very effective at identifying precisely the impact of the drug being tested on the ailment for which it is targeted, the sample size and study design don’t lend themselves to exploring a lot of additional factors.
Now consider electronic medical records. Over time, it will be possible to track years of medical history across millions of patients. Once a new drug comes out, these records can be utilized to mine for unexpected positive and negative drug interactions and unforeseen impacts on other health conditions. Certainly, the data won’t have all of the controls to ensure that correlations found are truly causal. However, if you’ve seen 50,000 people take a drug for 5 years and you see a recurrent pattern of those patients having a high incidence of some positive or negative experience, you can be fairly confident that something worth investigating further is going on.
While it may take additional work to confirm the findings, including perhaps a formal clinical trial, the ability to identify the right places to look will be increased exponentially. It isn’t that this data will replace clinical trials for new medications, but it can speed identification of additional risks or benefits of a medication after release. Historically, unless a drug had a massive side effect that wasn’t anticipated, it could take years for a positive or negative interaction to be identified. There was a dependency on hearing enough anecdotal feedback from doctors for the pattern to be found. With electronic medical records, it will be possible to analyze thousands or millions of people taking a drug from day one and identify good and bad side effects much more quickly. All of this assumes the medical data is anonymized for privacy reasons, of course.
The Implications
The point of the example above is to demonstrate how the way we view and execute analytics to assess a drug’s impact can change and improve time to value dramatically. In addition to the rigorous clinical trials, immense value can be found in the masses of data that, while not scientifically controlled, will contain enough information on enough patients with enough combinations of ailments and prescriptions to shine a light on unexpected effects. Utilizing data from an uncontrolled environment may not be how it is done today, but the power of the large samples of patients with diverse conditions is too much to ignore.
Within your own organization, be sure to look for opportunities to totally rethink how you apply analytics to important areas of your business. Are there ways that big data can allow you to come at a problem from a completely different direction to enhance what you’re doing today? In many cases, the answer is yes.
The action I’d like you to consider is to ensure that your conversations around big data don’t involve just how to use it within existing processes or how to use it to tackle entirely new problems. These two conversations are critical and can drive a lot of value. However, the third conversation you need to have, which many forget, is where big data can enable an entirely new approach to addressing an existing problem. A problem you’ve written off as “solved” may not be as solved as it used to be if you reexamine it in the context of the data available today.
Originally published by the International Institute for Analytics
