Comments by Paul Barsch Subscribe

On Preserving Big Data to Live Forever

Rob, wasn't that brief documentary fascinating? These folks are preserving data for generations to come. And their choices are way beyond which technologies to use, but also delving into risk management with a several hundred year vision. Amazing!

On Don't Gloat Over Excel Model Failures

Tonio, I appreciate your comments. R&R analysis was used by many as the intellectual basis for austerity. This is a real shame because R&R were careful to say that their analysis did not prove causation, only correlation. However,  R&R did not seem to stop anyone from asserting their analysis SHOULD NOT be used to promote austerity policies. And they should have.

I agree with the Anders Aslund quote from FT; “(While) the critique of Reinhart and Rogoff correctly identifies some technical errors in their work, one cannot read it and conclude the case for austerity is much weakened. High public debt is still a serious problem.” 

The key point I was trying to make – perhaps not very well—isn’t austerity vs. stimulus. It’s simply that we shouldn’t throw an idea out as false simply because someone has shown the maths supporting the idea are incorrect, or data skewed intentionally or not.  Here’s a perfect example:

http://www.skepticalscience.com/marcott-hockey-stick-real-skepticism.html

On Don't Gloat Over Excel Model Failures

Richard, R&R should have had someone review their results before publishing. No excuses. And I think the swing in data --as you mentioned---from barely negative to positive is significant.

That said, I'm not sure we're off the hook "intellectually" just because the data DO or DO NOT our hypothesis, especially in dealing with complex problems. That's because in terms of complexity, we often don't know all the variables, we don't know how to weight properly, and we may have incomplete data sets. Worse, our data sets (even if accurate) may not represent very well the fast changing conditions going forward.

Thanks for joining the discussion. I appreciate your input!

On Technologies and Analyses in CBS’ Person of Interest

Earl, that's a fun comment! I wonder if President Obama's $100m effort into mapping the brain will help accelerate things.  Skynet here we come! Thanks for commenting, I appreciate it.

On Technologies and Analyses in CBS’ Person of Interest

Anand, thanks for bringing these technology and analysis efforts up a level into some concrete categories.  I appreciate the thought that went into your comments and thanks for posting them!

On Data Scientist Scarcity: Automation Is the Answer

Radhika - thanks for this post.

Data scientists are scarce today, no doubt. But today's universities are vigorously training and adopting analytics programs as part of their core ciriculum for business and IT professionals. And they are also doubling down on statistics courses for graduates and undergraduates. CompSci degrees are hot and will be even hotter in the future. http://www.xconomy.com/seattle/2011/05/17/red-hot-the-computer-science-job-market/

In addition, pre-built libraries of various algo's (provided by numerous vendors) will help business analysts spend their time asking questions and analyzing the business instead of coding. 

In the next 3-5 years, these trends will help alleviate the shortage of Data Scientist skill sets, in addition to the automation angle discussed in your post.

On Big Data Success Stories: Take Them with a Grain of Salt

Tom, thanks again for commenting. As they say in the NFL, "upon further review" I did not intend the ratio to be seen as 1:1000, though upon reading it again, I can easily see how it's perceived as such. Thanks for pointing this out.

You also said something interesting - "Valid match of use case to technology and methodolgy. Do that and big data projects should NOT fail - period."  I do believe that use case is important and business case even moreso, with real financial projections. But to me, there are other variables that also affect big data success, including culture (willingness/or not to accept change), business sponsorship (or lack thereof), choosing the right vendor, sourcing the right skills, and even some of the softer stuff like communication.  All key factors in whether a Big Data project is successful, or joins the "thousands of failures". :)

Thanks again for adding your expertise. It's been a fun discussion and probably one of the more lively that SDC has seen!

On Big Data Success Stories: Take Them with a Grain of Salt

Thanks so much for commenting Meta!

I also found this survey on the web that might be beneficial for those reviewing this thread:

http://www.infochimps.com/resources/white-papers/cios-big-data

On Big Data Success Stories: Take Them with a Grain of Salt

Sebastien - good point! The term "Big Data" is not necessarily synonymous with Hadoop or other NoSQL data stores. Common mistake isn't it? In fact, if you look at the defintion of Big Data - as I previously commented - MPP databases have long met the "Volume" criteria in spades.

Thanks so much for commenting and adding to the discussion!

On Big Data Success Stories: Take Them with a Grain of Salt

Hi Tom, thanks for taking time from your busy schedule to comment.

I’d like to address some of your points.

First, there is no concerted PR agenda here. You give me much too much credit. I’m simply not that smart.  The opinions on SDC and my own blog are my own and I clearly state this on my own blog.

Second, I recognize that Big Data is a very nebulous term. The most common definition is the 3Vs coined by Doug Laney – Volume, Variety, Velocity. Also have seen a 4^th V for value added. I do not equate Big Data on a 1:1 basis with Hadoop or other NoSQL implementations.  Further, if you consider that many analytics implementations are of multiple TB scale (Volume), and consider the total failures of analytics projects, then I absolutely stand by my statement of “thousands of failures”.

Third, sometimes it’s hard to work all relevant concepts into a blog post of 500 words. The theme I was after is pretty simple actually: There’s lots of Big Data hype and for companies to be careful because in terms of cases and “Big Data Success” – one size does not fit all. In other words, as they say in the pharma space; “your results may vary.”

Fourth, I applied Nassim’s quote to my topic. True Nassim was not talking about Big Data when he spoke of graveyards of unseen observations. That said, if you want specifics of what Nassim does think of the topic of Big Data, you can Google “NYT”and “Big Data”. He actually has a scathing review of the topic.

Finally, I see that you work for IBM. I have no axe to grind with IBM. In fact, IBM is a terrific partner. And I must confess I am enamored with Watson. Absolutely enamored.

I’ll send you a Linkedin invite so we can continue the dialog. I always love to connect with extremely bright and talented people, so here’s hoping in advance that you’ll accept the invite.

Thanks again for commenting. I appreciate your insights.

On Dear Oracle: Cloud Multitenancy DOES Matter

Alok, really enjoyed this article. And will agree that multi-tenancy drives significant cost benefits for a good portion of customers. However, there are plenty of customers who are very wary of sharing anything - the same rack, cabinet, network equipment and more, much less the same server environment. For those customers (mostly those with security/privacy requirements), having their own server makes sense. And no amount of convincing about how a cloud provider is assuring security/privacy is going to allay their fears. They want what they want, and I think it's the smart cloud provider that gives it to them, although arguably at a higher cost.

Thanks for posting this, I like your writing style!

On Technologies and Analyses in CBS’ Person of Interest

Excellent find Meta. I have seen articles similar on various police forces across the United States. Thank you so much for commenting.

Btw, similar comment to yours, a colleague in my office mentioned I was missing geospatial/location intelligence in the mix of technologies/techniques.