We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: A Big Data Cheat Sheet: What Executives Want to Know
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Best Practices > A Big Data Cheat Sheet: What Executives Want to Know
Best PracticesBig DataData WarehousingHadoop

A Big Data Cheat Sheet: What Executives Want to Know

TamaraDull
TamaraDull
7 Min Read
A Big Data Cheat Sheet: What Executives Want to Know
Illustration generated with Qwen Image.
SHARE

In April, I was given the opportunity to present An Executive’s Cheat Sheet on Hadoop, the Enterprise Data Warehouse and the Data Lake at the SAS Global Forum Executive Conference in Dallas. During this standing-room only session, I addressed these five questions:

Contents
  • 1. What can Hadoop do that my data warehouse can’t?
  • 2. We’re not doing “big” data, so why do we need Hadoop?
  • 3. Is Hadoop enterprise-ready?
  • 4. Isn’t a data lake just the data warehouse revisited?
  • 5)  What are some of the pros and cons of a data lake?
  1. What can Hadoop do that my data warehouse can’t?
  2. We’re not doing “big” data, so why do we need Hadoop?
  3. Is Hadoop enterprise-ready?
  4. Isn’t a data lake just the data warehouse revisited?
  5. What are some of the pros and cons of a data lake?

Following is a recap of my comments, along with a few screenshots. See what you think.

1. What can Hadoop do that my data warehouse can’t?

The short answer is: (1) Store any and all kinds of data more cheaply and (2) process all this data more quickly (and cheaply).

The longer answer is: I made reference to my opening “soapbox” statement – “Big data is not new.” (See image below.) They say that 20% of the data we deal with today is structured data (see orange-box examples). I also call this traditional, relational data. The other 80% is semi-structured or unstructured data (see blue-box examples), and this is what I call “big” data. 

More Read

Should You Divorce Your Customer? CRM Analytics Can Tell You If It’s Time to Split
Should You Divorce Your Customer? CRM Analytics Can Tell You If It’s Time to Split
Ramifications of IT Infrastructure Everywhere
Role of Big Data Extraction for INTL Contextual Marketing Campaigns
Is Predictive Analytics Changing The Future Of Mobile Phone Monitoring?
How To Create A 360-Degree Customer View Using Data

Image

Are any of these blue-box example data types new? Of course not. We’ve been collecting, processing, storing, and analyzing all this data for decades. What we haven’t been able to do very well, however, if at all, is mix the orange- and blue-box data together.

So here’s what’s new: We now have the technologies to collect, process, store, and analyze all this data together. In other words, we can now mix-&-match the orange- and blue-box data together – at a fraction of the cost and time of our traditional, relational systems.

2. We’re not doing “big” data, so why do we need Hadoop?

I proposed six common Hadoop use cases—three of which don’t require “big” data at all to take full advantage of Hadoop. These use cases come from my white paper called The Non-Geek’s Big Data Playbook: Hadoop and the Enterprise Data Warehouse.

Here’s a brief summary of each use case:

  • Stage structured data. Use Hadoop as a data staging platform for your data warehouse.
  • Process structured data. Use Hadoop to update data in your data warehouse and/or operational systems.
  • Archive all data. Use Hadoop to archive all your data on-premises or in the cloud.
  • Process any data. Use Hadoop to take advantage of non-integrated and unstructured data that’s currently unavailable in your data warehouse.
  • Access any data (via data warehouse). Use Hadoop to extend your data warehouse and keep it at the center of your organization’s data universe.
  • Access any data (via Hadoop). Use Hadoop as the landing platform for all data and exploit the strengths of both the data warehouse and Hadoop.

If you’d like to see these use cases further explained and demonstrated with some easy-to-understand visuals, I invite you to download the white paper.

3. Is Hadoop enterprise-ready?

I have two answers to this question:

  • For your organization: Maybe.
  • For all organizations: No.

It all depends on what and how you want to use Hadoop in your organization. If you simply want to use it as an additional (or alternative) storage repository and/or as a short-term data processor, then by all means, Apache Hadoop is ready for you.

However, if you want to go beyond data storage and processing and are looking for some of the same data management and analysis capabilities you currently have with your existing relational systems, you will first need to explore the vast ecosystem of Hadoop-related open source and proprietary projects and products. This will not be a small undertaking.

Because many of these newer Hadoop-related technologies are still maturing—quite rapidly, I might add—that’s why I say Hadoop—as in the Hadoop ecosystem—isn’t 100% ready for the enterprise.

4. Isn’t a data lake just the data warehouse revisited?

Many of us have been learning more about the data lake, especially in the last 6 months. Some suggest that the data lake is just a reincarnation of the data warehouse—in the spirit of “been there, done that.” Others focus on how much better this “shiny, new” data lake is, while others are standing on the shoreline screaming, “Don’t go in! It’s not a lake—it’s a swamp!”

All kidding aside, the commonality I see is that they are both data storage repositories. Beyond that, the table below highlights some key differences. This is, by no means, an exhaustive list, but it does get us past this “been there, done that” mentality. A data lake is not a data warehouse.

Image

5)  What are some of the pros and cons of a data lake?

Some of you may be aware of the Data Lake Debate blog series I recently participated in with my colleague, Anne Buff, on SmartData Collective. I took the Pro stance, Anne took the Con stance, and our boss, Jill Dyché, moderated.

It was an intense 8 weeks of discussion—loosely structured like a Lincoln-Douglas debate—and many key points about the data lake were addressed. During my presentation, I summed up these key points using a SWOT diagram:

Image

If you’re interested in learning more about the data lake, I invite you to:

  • Check out the Data Lake Debate blog series; and/or
  • Register for the Data Lake Debate webcast on May 27th where Anne and I will “go live” with the debate.

And there you have it—your big data cheat sheet. Please share with others if you’ve found it helpful. Thanks!

TAGGED:The Big Data MOPS Series
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive
Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

PII, Anonymized Data, and Big Data Privacy
Big DataPrivacySocial Data

PII, Anonymized Data, and Big Data Privacy

5 Min Read
Introducing the Big Data MOPS Series
Best PracticesBig Data

Introducing the Big Data MOPS Series

6 Min Read
Connecting the Data Dots Keeps These Companies Alive
Best PracticesBig DataData ManagementKnowledge Management

Connecting the Data Dots Keeps These Companies Alive

6 Min Read
If You Think Data Security is IT’s Responsibility, Think Again
Best PracticesBig DataSecurity

If You Think Data Security is IT’s Responsibility, Think Again

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?