What do brilliant data scientists, roller derby enthusiasts, and Prince William lookalikes have in common? You can find all three at Kaggle, a company whose mission is to bring together the top data scientists in the world to solve complex problems. Kaggle draws from an impressive pool of tens of thousands of PhDs who compete against one another to create predictive models with extraordinary capabilities. Their clients include some serious heavyweights—Microsoft, MasterCard, and even NASA have hosted competitions.
I recently had the opportunity to chat with Anthony Goldbloom, the company’s founder and CEO, and he kindly shared some insights about his company, data and analytics, and his experiences around the world.
Hailing from the land down under, Goldbloom graduated with first-class honors in Economics and Econometrics from the University of Melbourne. However, don’t let his laid back Australian accent fool you; he is one of the most driven and forward-thinking people in San Francisco today. Kaggle was founded in 2010 after Goldbloom spent a summer interning for the Economist. He recognized the global need for scientists with a good understanding of data and statistics. However, he also realized that most companies had trouble finding and recruiting these unique individuals.
So what do you do when the desired human capital is scattered all around the world? Goldbloom’s answer was to offer incentives and turn the problem into a competition for the highest caliber of scientists. Over 90,000 PhDs, industry researchers and freelancers gather on Kaggle to give their best at modeling solutions to complicated and often urgent problems. Their fields of expertise include energy, healthcare, manufacturing, consumer goods, technology, financial services and more. While the US is a powerhouse in big data and analytics, Eastern Europe and Asia are quickly catching up in their quality data scientist output.
The problems that Kaggle solves are highly relevant and often ones that plague us the most. If you have had the misfortune of flying from busy airport hubs such as O’Hare International in Chicago or London Heathrow, you can understand why flight delays and accurate reporting are a pressing issue for many. In 2012, Kaggle worked with GE to generate a predictive model that better anticipates flight arrival time. By analyzing weather conditions, the location of the plane, flight traffic and travel time, scientists created a model that showed a 40% improvement in accurate predictions.
Another competition Kaggle hosted with the Hewlett Foundation, a non-profit organization, resulted in the creation of a program that graded students’ essays with the same accuracy and reliability as teachers. With this technology, teachers have more time to spend on creating or enhancing material for their classrooms. This is just one of the many projects that Kaggle scientists take on in order to better our world.
Kaggle not only promotes competitions, but the company also offers Kaggle Connect, a consulting platform that connects companies to elite data scientists. Anyone with an interesting problem and dataset can buy hours from Kaggle Connect. Then through the power of selective crowdsourcing, data analysts extract insight from the data using their individually unique methods. It’s an affordable way for businesses to try out the hype they’ve been hearing about big data and analytics and see if it’s right for their company.
After hearing about all the projects they take on, I asked Anthony what he saw as the most interesting or challenging one. His answer? “The most challenging thing is that a lot of people don’t have a good sense of what data can be useful for. Therefore, we spend a lot, and I mean a lot of, time educating others on what is possible. This is indicative of an expanding market for data and predictive modeling.” What motivates him when he wakes up each morning is the idea of getting more people to use data science, and democratizing decisions that are made based on hard data and not just intuition or hunches.
“I have a great vision for the future, where the majority of decisions are made on the basis of data.” –Anthony Goldbloom
As someone who has countless interactions with people talking about big data, business analytics and predictive modeling, I am refreshed by Kaggle’s approach to it all. It’s a company poised on the forefront of the data scene that will revolutionize how businesses solve problems. Goldbloom talked about a great vision for the future, but this future is already here and it’s moving forward fast.
Goldbloom will be attending the Data 2.0 Summit in San Francisco on April 30, 2013 as a featured speaker.