Big or Small
I can see a large number of advantages for SME’s in using big data technology on small and private data sets but are there also many reasons for them to look to Big Data itself. A recent survey we ran found that it is the top priority for SME’s, but first of all; what is big data?
Big or Small
I can see a large number of advantages for SME’s in using big data technology on small and private data sets but are there also many reasons for them to look to Big Data itself. A recent survey we ran found that it is the top priority for SME’s, but first of all; what is big data?
SAP define it as “a popular term used to describe the exponential growth and availability of data, both structured and unstructured. And big data may be as important to business – and society – as the Internet has become. Why? More data may lead to more accurate analyses.” (It strikes me as very SAP like to say ‘More data may lead to more accurate analyses’!)
Amongst the early descriptions that were availible, Gartner defined Big data as; “three V’s; high Volume, high Velocity, high Variety”, and this is one way of looking at what Big Data is, as is considering it as ‘large or complex data sets’. If you are interested in what other definitions of big data is there are a further 26 interesting definitions here. My own definition is; Big Data is a series of combined data sets that require a different toolset from the traditional relational database management systems RDMS. I also like the slightly poetic notion of ‘Data Lakes’.
No matter which definition is used, I am not alone in seeing many advantages for SME’s looking to big data to gain insights and the following comments all sit very well with this view.
How the Analytics industries see SME’s fitting into the Big Data picture.
According to Mark van Rijmenam of Datafloq “data is being created faster than ever before and small and medium businesses will reach a stage where the cost of storing that data with traditional databases is becoming too expensive.” he believes that SMEs should turn to big data storage solutions and that while “… it is true that some small or medium enterprises might not have that much data but they operate in supply chains and have suppliers or distributors and as they work together and share their data, the available data for analyses increase many times … Nike for example shares data from all its suppliers with the rest of the industry.” When SME’s start using and combining data from its suppliers and vendors, suddenly they have sufficient data to analyse, visualize and use for improved decision making. 1
Dave Becerra, vice-president of strategy and business development at Roambi, says “SMEs are beginning to see big data as something more than just an enterprise trend. He believes some are starting to realise that they can identify trends, patterns and gain competitive advantage by harnessing the power of growing data volumes.”
Lauren Walker, big data/analytics leader at IBM UK & Ireland, is emphatic that SMEs should be looking at big data. “Interest in big data has reached new heights for many SMEs as they attempt to capture information and glean insights from ongoing conversations on social channels and the ‘digital dust’ consumers leave when browsing the web, shopping online, listening to music in the cloud and using smartphone apps,”. 2
A recent report by Research and Markets forecasts “that the global big data market for small and medium-sized enterprises (SMEs) will grow at a compound annual rate of 43 per cent until 2018. Just like large enterprises, SMEs can use big data to better understand their customers, tap into new markets and cut out unnecessary costs across the business, all in real-time.” 3
Peter Simons believes that “With big data and analytics generally considered the domain of the giant corporation, many SME business owners end up overlooking the numerous opportunities these tools can provide for small and medium-sized enterprises. Unfortunately, such an oversight can result in SMEs losing out to competitors who are effectively using data as a means of improving performance and gaining new insights. “4
Alex Barrett at TechTarget points out that; “While many businesses see the benefit of running big data in public clouds, others look to nix infrastructure altogether for their analytics projects. Analytics as a Service creates hunches, bridges IT gaps …
cloud’s biggest contribution to solving the big data problem is the number of analytics vendors that have adopted the Software as a Service (SaaS) model. IT departments not only don’t need to buy infrastructure but also don’t need to set anything up … even for modest workloads, public cloud’s pay-per-use model is appealing. ‘The cloud is good at spin-up and spin-down,’ said Frances Guida, Hewlett-Packard’s manager for cloud in its enterprise group. That’s a nice fit with big data. ‘A lot of analytics aren’t predictable, and when you answer the question, you don’t necessarily need the infrastructure again’ .”5
Other ways SMEs will benefit from looking at Big Data Technology
Going back to one of the definitions of big data (high Variety) we see that nearly all SMEs are hampered by having pools of data in different silos. Due to the variety of data sources and the lack of a shared structure these pools of data are not easily consolidated using traditional tools, many hours cutting and pasting between spreadsheet’s is often the only method employed, and many medium sized enterprises spend a lot of their resouces doing just that. It is here that big data tools are so well equipped to turn complex information into simple pieces of knowledge. They allow you to collect the pools into a lake and fish out the bits of knowledge you need. Big Data tools can do this becuase they are designed for working with subsets of semi structured data, they do not have to worry too much about getting all of the data sets to fit into a schemaor more importantly have to change a scheme to accept the data.
Big data tools also bring other tricks to the table, machine learning and the ability to perform predictive analysis are found within this bag of tricks as are very flexible and powerful search techniques. Another trick is being able to move data into less expensive places, it can be brought into an area where it is examined and analysed and once some conclusions are found the examination and analysis resources can be put to sleep while the raw data is parked in a cheaper location. The ability to dive in and grab what you want allows access to very powerfull technology while paying only for the time you are in there exploring.