(An earlier version of this article was published at http://www.constellationrg.com/blog/2011/12/musings-watson-why-healthcare)
(An earlier version of this article was published at http://www.constellationrg.com/blog/2011/12/musings-watson-why-healthcare)
I was able to attend the IBM Connect Summit in Stamford, CT a few months ago. Beside the usual opportunities to see and share with friends and colleagues (who are too numerous to mention here), I attended a few sessions about Watson and was able to have some one-on-one conversations with IBM people connected with the project. Let me first say that I think Watson is a wonderful invention. I don’t know of another company other than IBM that has the resources to put together a project like this. There are groundbreaking technologies in semantics, natural language processing, machine learning, ontologies and a host of others all integrated to provide a single working entity. There are, I believe, over 40 subsystems in Watson, all seemingly working in harmony. It’s much noted triumph over the two winningest Jeopardy contestants was a nice coming out test, but IBM has much bigger, and more important things in mind.
Which brings me to my first point. Why health care? When I asked Karen Parrish, VP SWG Industry Solutions, a thoughtful and articulate executive this question, she said because health care is the most information-rich subject area. I haven’t measured information quantity by subject area, but even if health care isn’t number one, you can’t argue with her premise. There is a lot of stuff out there.
Watson is “trained” by consuming almost unimaginable amounts of information – every piece of medical information – books, journals, blogs, magazines and newspaper articles, published dissertations, presentations at professional conferences…everything, and he has total and complete recall of every word. Also Watson will never forget what he’s recommended and will compare that to outcomes (provided they find their way back). For specific applications, Watson will receive EMR (electronic medical records) of patients, and if the client is a payer, like an insurance company, all of the claims experience as well. I’m sure there is more and this is just what I’ve heard so far.
But imagine if a person had the same supernatural abilities as Watson, but no formal credentials and no clinical practice experience. Would that person be given privileges to practice medicine in a hospital? Now suppose that person wasn’t a person at all, but a machine that, no matter how magnificent its design and implementation in this age, did not possess a shred of human intelligence – intuition, empathy or even direct-to-human experience. The concept of “experience-based medicine” has been a reaction to doctors shooting from the hip without data, but is Watson a bridge too far?
IBM responds that Watson won’t be practicing medicine it will merely act as an intelligent advisor, offering diagnoses and treatment recommendations only. Let me add a personal note. My wife is the inventor of a series of patented medical protocols that are used around the world. Sometimes, patients on these protocols will contact her directly looking for advice. She is not a medical doctor, but certainly has more experience and insight into the basic research and the clinical practice of these protocols than most doctors who employ them. She always responds by saying she will contact the person’s physician and ADVISE them. In practice, 99 times out of 100, the doctor takes her advice. So this notion that Watson is only an advisor will have to be evaluated in practice.
This initial project is funded by WellPoint, which is not a medical practice, it’s an insurance company. It doesn’t dispense medical advice. It pays claims. My guess is Watson will assist WellPoint finding ways to reduce claims costs rather than being a friendly assistant to a Marcus Welby-like doctor in a starched white coat. It’s more likely Watson will be churning through thousands of claims and authorizations a day with no one curating its advice. This is the part that worries me.
The idea of diagnostic medical experts is not new. In the early days of (overhyped, oversold, underperforming Artificial Intelligence (AI)), a few of these were attempted but were not widely accepted because of their limited effectiveness. Actually, the problem was that they were not able to teach themselves like Watson does, and the job of building the medical knowledge base was too overwhelming. Here is a good historical piece from almost 25 years ago (I wonder what Watson thinks of it?):
SCHWARTZ WB, PATIL RS, SZOLOVITS P. Artificial intelligence in medicine: where do we stand? N Engl J Med. 1987;316:685-8.
Remember, Watson doesn’t “think” (that’s funny, if any of you are old enough to remember IBM’s use of the term over the years http://www-03.ibm.com/ibm/history/exhibits/vintage/vintage_4506VV2024.html). Watson shreds natural languages and uses quantitative, statistical and probabilistic techniques to create knowledge maps that it can score in order to offer ranked recommendations. This was more than adequate for playing Jeopardy, but Jeopardy is a parlor trick next to the health project. Don’t get me wrong, IBM is taking this seriously. There is an army of brilliant people working on this project. But like everything else we do in technology, it’s not the tool, its how its put into practice.
One suggestion I’ve made is, because medical research and journal reporting in the US is wildly stilted by the research and academic paper chase (watch for a future blog on this topic), all of it needs to be taken with a grain of salt. One simple solution would be to weight journal articles by the credibility of the Principle Investigators: are they on a pharmaceutical company speaker bureau? Did a funding grant to an academic institution come from a commercial concern that has skin in the game (almost always). Most importantly of all, are they reproducible? An alarming number, when tested by commercial labs (in-house pharmaceutical drug discovery, e.g., which DO NOT typically publish), are not. Does Watson make an attempt to seek out research that was not conducted by scientists connected to Harvard, Stanford, etc.? Many smaller journals publish excellent science from lesser-known institutions.
I suggested that Watson should also consume information on alternative medicine and medical practices in other countries, particularly Asian countries. It would help to read patient-advocate blogs. He has lots of time and is a pretty fast reader. But maybe this is not a concern at all. I understand that if Watson reads a Wikipedia article, he will follow the links if that’s relevant. But that’s the rub. What criteria are at play to decide what is relevant? That concerns me. When it comes to science, Wikipedia administrators are pretty tough on this topic of “reliable source” but in medicine, the reliable source pipe is flooded with the standard of care and research from about a dozen large, well-funed institutions that have a vested interest in not upsetting the apple cart.
Despite all my concerns and doubts, I truly hope IBM is successful with this. To put it in perspective, in the US, 40,000 women die of breast cancer each year, a horrible toll that takes women out in the prime of their lives. It gets lots of attention. It has its own ribbon color. But the estimates of deaths from iotragenic disease, death caused by medicine, doctors and hospitals range from 150,000 to as many as 800,000. There is no ribbon for iotragenic disease and very few people even know it exists. If Watson can make a dent in this, that would be a real achievement, but if he only reads the “standard of care,” he’ll be a contributor to it.
Someone needs to curate this information. For example, the National Cancer Institute (arguably more of a lobbying organization for the caner industry than anything else) complies tons of cancer statistics. I’m sure you read in the media for the last few years how we’re making steady progress “beating” cancer.” On what basis can they make this claim? Even M.D. Anderson’s president was quoted just the other day that less than 5% of new drugs for cancer make it through clinical trials. What force is at work to lower the cancer death rate? I don’t believe the lower death rates. Look how the industry releases cancer death statistics: “Cancer deaths: The estimated numbers of US cancer deaths are calculated by fitting the numbers of cancer deaths for 1969-2007 to a statistical model that forecasts the numbers of deaths expected to occur in 2011.” Seriously? You measure your progress by fabricating the numbers?
Lets use our heads on this one. If something happened to cause lower cancer death rates, historical trend analysis would tell us nothing at all. There would be a CHANGE in the trend. If anything, historical statistics would dampen the effect of any good news.
I’ll be blogging some more on the subject of cancer statistics. For now, just think about how Watson will deal with tainted data. My father told me a long time ago, “Don’t believe anything you hear and only half of what you read.”
(An earlier version of this article was published at http://www.constellationrg.com/blog/2011/12/musings-watson-why-healthcare)