Data is only as good as the way people comprehend it and put it to work. The field of data visualization presents data in forms that make more sense. One of the pioneers in modern data visualization is Tableau Software, which was spun out of the work of Pat Hanrahan as his team of Ph.D.’s at Stanford University. Now their products are being used by many industries and organizations. Ellie Fields is Tableau’s Director of Product Marketing. This interview was done over the phone.
SDC: Like most people we learn about graphs and charts and such, but the data scene today has taken us many levels up. A pie chart or bar graph can help you understand numbers by providing a basis for visual comparison, but data today is many levels more complicated than what what we’ve depicted in the past. What does visualization, as you provide at Tableau, tell people that they can’t see just by looking a comparisons of numbers?
Fields: There’s a couple of different ways to answer that question, and I’ll start at the most basic and go into some of the things that are happening today.
Visualization science grew in the ’50s and ’60s with Tuffte and those folks. And basically what it says is that numbers are essential – they’re great – but the human brain – I don’t care how smart or analytical you are – the human brain just doesn’t process large sheets of numbers.
Take any computer science Ph.D. and put five pages of rows and numbers in front of them and they’re not going to very quickly understand what’s in there. Our brains just don’t work that way. Our brains work in stories and they work in pictures. What data visualization allows you to do is take a whole bunch of numbers and tell stories with it. There’s almost never just one story in a dataset. Different people might have different stories they care about.
A great example is business data. A sales person might really care about how sales are trending for the quarter. A product person might care about which products are selling more and accelerating so they know what to build in the future. A financial person might care about what they’re seeing on the bottom line this quarter. So you can take any one set of numbers and tell a variety of stories with it. We tell those stories through pictures.
You can think of the standard business dashboard as really telling a story of what’s going on in the business right now, but there are deeper levels to it as well. Stories, if they’re good, let people drill down into numbers and tell their own stories and let them ask and answer questions. Those are some of the things that interactive visualization lets you do.
First you’ve just got to be able to take those numbers, find what’s interesting. We often talk about “trends and outliers” because if you took any dataset and tried to visualize every single number in it, you’d probably be back to a bunch of rows and columns and it wouldn’t be very interesting. But find the trends, find the outliers, find the interesting things for the people who care about that part of the story, and then present that data and let them browse around; let them go through time; let them drill down through a product category to specific products; let them drill down into manufacturing plants that build those products.
Those are the kinds of things you can start to do with interactive visualization, and then you get into really powerful ways to use your data. So that’s the high level theory of visualization.
SDC: What is Tableau doing to address this reality of human perception?
Fields: Our mission is to help people to see and understand the world’s data. That hasn’t changed since we were spun out of Stanford. It’s that big that there’s that much data out there and people need to see and understand it.
That’s kind of the big picture mission. What we see happening today in the industry is an acceleration towards more data everywhere. If you think about any given company, they have so much more data than they had 20 years ago, even things like CRM systems.
SDC: What has changed in the way workers need and use data?
Fields: Before, you had a bunch of sales people with their notes about who they called and their own agenda books and so on. Now you can get your data out of Salesforce and Microsoft CRM and a bunch of other systems. And you can actually do analytics on how many times you call your customers, which ones are converting out of different industries, which ones aren’t converting, which sales people are doing well, which salespeople aren’t…there’s just a variety of numbers that didn’t exist before.
The online world is another great example. Web analytics didn’t exist 20 years ago and now companies have so much analytics – things like e-commerce, what transactions are happening on your website, retail data, RFID…I could go on and on.
SDC: How is the increase in data driving the development of visualization technology?
Fields: That proliferation of data makes visualization even more important because if our brains couldn’t comprehend five pages of rows and numbers before, there’s no way we’re going to comprehend thousands of pages of rows and numbers, which is effectively what a lot of companies have. And so they’re kind of on this curve – companies have gotten to the point where they say, “Data’s important, let’s collect data,” and they put it into a warehouse or cube or database – and governments are doing this too – and the data is there now. And the link that hasn’t happened is they aren’t telling the stories with that data. I would argue that most of the world’s data is still sitting collected in a nice or not-so-nice location, in terms of how accessible it is, but there are tons of stories that need to be mined in there.
SDC: So do people have to learn how to extract and tell those stories? Is that a skill that is lacking?
Fields: I think working with data is a skill that a lot of people are developing. Our philosophy on that is that anybody who understands their business or domain ought to be able to work with their data.
So if I’m running a manufacturing plant, I shouldn’t need to, for example, be an IT expert. I shouldn’t have to know how to configure databases. I shouldn’t need to be a programmer. I shouldn’t need to know how to do SQL queries. I shouldn’t need to acquire specialized knowledge to just get a simple questions about my own business, like which machines are running, what’s making money, what’s not. I should just be able to connect to the data that lives somewhere, and ask it questions in a fluid, drag and drop way.
And that’s what we’re trying to do with visualization. There are a lot of tools out there that – if you have spent years getting a degree in something – you can use to create some pretty cool stuff. But business is changing fast, and every time your business changes or a new competitor comes on the scene or you open a new region, if you have to go back to an IT expert and ask, “Please, pretty please, can you change my dashboard so I can see what’s happening in my business?” – which is how most businesses end up doing it today – we hear from people that it can take six months to make those changes.
So if you think about reacting to that business environment – which is exactly what we have today – six months just to get at the new data that you already have – to see a new view of it – is a long time to wait. Even if it’s only a month or two months, that’s still a long time.
What we think is that if you have the security credentials to access a database – IT says, “Yeah, you can look at that data, you’re an authorized user” – you ought to be able to go and make your own query and do it in an intuitive drag and drop way so you don’t have to go and learn programming to do that.
SDC: How does Tableau help companies deal with unstructured and dirty data? How does that get portrayed through visualization?
Fields: That’s a good question. We can connect to any kind of data that’s structured, and almost all data—even structured data– is dirty, so that’s something we deal with all the time. Then with dirty data we have tools that let you clean up your data inside the tool by excluding values that are obviously false or recoding things that are mis-coded…it all depends on the data, but there are ways to deal with it at different levels.
Fields: There’s a couple of different ways to answer that question, and I’ll start at the most basic and go into some of the things that are happening today.
Visualization science grew in the ’50s and ’60s with Tuffte and those folks. And basically what it says is that numbers are essential – they’re great – but the human brain – I don’t care how smart or analytical you are – the human brain just doesn’t process large sheets of numbers.
Take any computer science Ph.D. and put five pages of rows and numbers in front of them and they’re not going to very quickly understand what’s in there. Our brains just don’t work that way. Our brains work in stories and they work in pictures. What data visualization allows you to do is take a whole bunch of numbers and tell stories with it. There’s almost never just one story in a dataset. Different people might have different stories they care about.
A great example is business data. A sales person might really care about how sales are trending for the quarter. A product person might care about which products are selling more and accelerating so they know what to build in the future. A financial person might care about what they’re seeing on the bottom line this quarter. So you can take any one set of numbers and tell a variety of stories with it. We tell those stories through pictures.
You can think of the standard business dashboard as really telling a story of what’s going on in the business right now, but there are deeper levels to it as well. Stories, if they’re good, let people drill down into numbers and tell their own stories and let them ask and answer questions. Those are some of the things that interactive visualization lets you do.
First you’ve just got to be able to take those numbers, find what’s interesting. We often talk about “trends and outliers” because if you took any dataset and tried to visualize every single number in it, you’d probably be back to a bunch of rows and columns and it wouldn’t be very interesting. But find the trends, find the outliers, find the interesting things for the people who care about that part of the story, and then present that data and let them browse around; let them go through time; let them drill down through a product category to specific products; let them drill down into manufacturing plants that build those products.
Those are the kinds of things you can start to do with interactive visualization, and then you get into really powerful ways to use your data. So that’s the high level theory of visualization.
SDC: What is Tableau doing to address this reality of human perception?
Fields: Our mission is to help people to see and understand the world’s data. That hasn’t changed since we were spun out of Stanford. It’s that big that there’s that much data out there and people need to see and understand it.
That’s kind of the big picture mission. What we see happening today in the industry is an acceleration towards more data everywhere. If you think about any given company, they have so much more data than they had 20 years ago, even things like CRM systems.
SDC: What has changed in the way workers need and use data?
Fields: Before, you had a bunch of sales people with their notes about who they called and their own agenda books and so on. Now you can get your data out of Salesforce and Microsoft CRM and a bunch of other systems. And you can actually do analytics on how many times you call your customers, which ones are converting out of different industries, which ones aren’t converting, which sales people are doing well, which salespeople aren’t…there’s just a variety of numbers that didn’t exist before.
The online world is another great example. Web analytics didn’t exist 20 years ago and now companies have so much analytics – things like e-commerce, what transactions are happening on your website, retail data, RFID…I could go on and on.
SDC: How is the increase in data driving the development of visualization technology?
Fields: That proliferation of data makes visualization even more important because if our brains couldn’t comprehend five pages of rows and numbers before, there’s no way we’re going to comprehend thousands of pages of rows and numbers, which is effectively what a lot of companies have. And so they’re kind of on this curve – companies have gotten to the point where they say, “Data’s important, let’s collect data,” and they put it into a warehouse or cube or database – and governments are doing this too – and the data is there now. And the link that hasn’t happened is they aren’t telling the stories with that data. I would argue that most of the world’s data is still sitting collected in a nice or not-so-nice location, in terms of how accessible it is, but there are tons of stories that need to be mined in there.
SDC: So do people have to learn how to extract and tell those stories? Is that a skill that is lacking?
Fields: I think working with data is a skill that a lot of people are developing. Our philosophy on that is that anybody who understands their business or domain ought to be able to work with their data.
So if I’m running a manufacturing plant, I shouldn’t need to, for example, be an IT expert. I shouldn’t have to know how to configure databases. I shouldn’t need to be a programmer. I shouldn’t need to know how to do SQL queries. I shouldn’t need to acquire specialized knowledge to just get a simple questions about my own business, like which machines are running, what’s making money, what’s not. I should just be able to connect to the data that lives somewhere, and ask it questions in a fluid, drag and drop way.
And that’s what we’re trying to do with visualization. There are a lot of tools out there that – if you have spent years getting a degree in something – you can use to create some pretty cool stuff. But business is changing fast, and every time your business changes or a new competitor comes on the scene or you open a new region, if you have to go back to an IT expert and ask, “Please, pretty please, can you change my dashboard so I can see what’s happening in my business?” – which is how most businesses end up doing it today – we hear from people that it can take six months to make those changes.
So if you think about reacting to that business environment – which is exactly what we have today – six months just to get at the new data that you already have – to see a new view of it – is a long time to wait. Even if it’s only a month or two months, that’s still a long time.
What we think is that if you have the security credentials to access a database – IT says, “Yeah, you can look at that data, you’re an authorized user” – you ought to be able to go and make your own query and do it in an intuitive drag and drop way so you don’t have to go and learn programming to do that.
SDC: How does Tableau help companies deal with unstructured and dirty data? How does that get portrayed through visualization?
Fields: That’s a good question. We can connect to any kind of data that’s structured, and almost all data—even structured data– is dirty, so that’s something we deal with all the time. Then with dirty data we have tools that let you clean up your data inside the tool by excluding values that are obviously false or recoding things that are mis-coded…it all depends on the data, but there are ways to deal with it at different levels.
But with unstructured data we see customers do different things. Sometimes they do some analysis beforehand. For instance, we have a customer, Ernst and Young, who does a lot of fraud analysis with email – they’re trying to find patterns in email – that might lead to some fraudulent activity. They do that for their clients, and they take a lot of that data and create some structure around it through word searches and incidents and then they analyze that in Tableau.
SDC: I assume, then, that your software reacts in real time so that people can follow those changes. Hans Rosling uses animated data visualizations to portray changes over time. Does Tableau do that?
Fields: Yes, it does, using a mechanism we call the Page Shelf and you can basically animate through time or animate over a region. Right now we don’t have that in the online version, but in the desktop version you can do that.
SDC: Where is this leading? What’s your cutting edge in terms of developing this technology. There’s a lot of change happening, as judged by the energy around big data conferences we’ve attended. Where’s the great challenge for Tableau?
Fields: We’re developing along three major thrusts right now. The first is in richness of visualization. There’s always more you can do in this area. One example of that is network diagrams. Network data didn’t used to be that useful. You had to have a social scientist go in and map it out to get network data, but now we have network data all over the place. So people are very interested in mapping networks among their customers or social media followers or what have you. That’s just one example of how we’re thinking of pushing the envelope in visualization. Everything having to do with visual analysis data we want you to be able to do in Tableau.
Another big thrust is data and dealing with bigger and bigger data, so one of the things we’ve come out with recently is an in-memory data engine that lets you – if you’re working with a lot of data and it’s very slow like in a transactional database – you can pull it into memory on commodity hardware that people carry around with them in their laptop bags today, and you can do quite a sophisticated analysis on hundreds of millions of rows of data using the memory right on your machine. We’ve come out with some technology that lets you do that and be very fast with data.
There are a number of ways we’re going to continue to innovate. Big data is one. We’ve come up with some tools that lets you mash up different data sets without having to do integration work, so you can look at your financial data and product data if they live in two different databases together.
And we’re trying to stay on the leading edge following developments like Hadoop and other new technologies that are coming out to deal with big data.
The last thrust that we’re looking at very seriously – we’re in beta now and we’re coming out with this in about a month – is mobile, because business intelligence is this old industry that has a lot of bugaboos.
This 30-year-old industry is not very heavily adopted. Almost every major company has bought business intelligence software, and in many cases they’ve bought a lot of licenses. And TDWI did a study that showed that only about 8% of companies use BI, and that’s a huge failing because if you try to build a data-driven culture of decision making, it’s hard if people aren’t even looking at the data. And so, we think a lot of the things we’re doing will help more people drive business intelligence adoption simply by making things easier to use, but mobile is a big way to do that, too, because where decisions happen are often in conference rooms and on manufacturing floors and on customer sites and if you have to go home and sit at your desktop or your laptop to actually get the data to help you make a decision right then, it’s not going to happen in most cases. So mobile, in many ways, fulfills the promise that was first laid out by this very old industry.
SDC: I assume, then, that your software reacts in real time so that people can follow those changes. Hans Rosling uses animated data visualizations to portray changes over time. Does Tableau do that?
Fields: Yes, it does, using a mechanism we call the Page Shelf and you can basically animate through time or animate over a region. Right now we don’t have that in the online version, but in the desktop version you can do that.
SDC: Where is this leading? What’s your cutting edge in terms of developing this technology. There’s a lot of change happening, as judged by the energy around big data conferences we’ve attended. Where’s the great challenge for Tableau?
Fields: We’re developing along three major thrusts right now. The first is in richness of visualization. There’s always more you can do in this area. One example of that is network diagrams. Network data didn’t used to be that useful. You had to have a social scientist go in and map it out to get network data, but now we have network data all over the place. So people are very interested in mapping networks among their customers or social media followers or what have you. That’s just one example of how we’re thinking of pushing the envelope in visualization. Everything having to do with visual analysis data we want you to be able to do in Tableau.
Another big thrust is data and dealing with bigger and bigger data, so one of the things we’ve come out with recently is an in-memory data engine that lets you – if you’re working with a lot of data and it’s very slow like in a transactional database – you can pull it into memory on commodity hardware that people carry around with them in their laptop bags today, and you can do quite a sophisticated analysis on hundreds of millions of rows of data using the memory right on your machine. We’ve come out with some technology that lets you do that and be very fast with data.
There are a number of ways we’re going to continue to innovate. Big data is one. We’ve come up with some tools that lets you mash up different data sets without having to do integration work, so you can look at your financial data and product data if they live in two different databases together.
And we’re trying to stay on the leading edge following developments like Hadoop and other new technologies that are coming out to deal with big data.
The last thrust that we’re looking at very seriously – we’re in beta now and we’re coming out with this in about a month – is mobile, because business intelligence is this old industry that has a lot of bugaboos.
This 30-year-old industry is not very heavily adopted. Almost every major company has bought business intelligence software, and in many cases they’ve bought a lot of licenses. And TDWI did a study that showed that only about 8% of companies use BI, and that’s a huge failing because if you try to build a data-driven culture of decision making, it’s hard if people aren’t even looking at the data. And so, we think a lot of the things we’re doing will help more people drive business intelligence adoption simply by making things easier to use, but mobile is a big way to do that, too, because where decisions happen are often in conference rooms and on manufacturing floors and on customer sites and if you have to go home and sit at your desktop or your laptop to actually get the data to help you make a decision right then, it’s not going to happen in most cases. So mobile, in many ways, fulfills the promise that was first laid out by this very old industry.
