The Insight 2015 conference highlighted some impressive applications of big data: predicting the path of hurricanes more accurately (as we saw with hurricane Patricia), improving the performance of athletes, making cars safer, etc.
These applications involve large amounts of data. But more importantly they involve new data, not simply greater quantities of data we’ve had before. Cheap sensors make it possible to measure things more directly and in higher resolution than before. We have sources of data, such as social media, that are qualitatively different from what we’ve had in the past.
Simply saying we have more data than before obscures what’s happening. For example, we don’t know more about consumer behavior than a generation ago because we do more phone surveys and have more customer satisfaction post cards to fill out. We know more because we can observe things we couldn’t observe before.
Clever analysis deserves some credit for the successes of big data, but more credit goes to new sources of data and the technologies that make these sources possible.
N and M in the matrix of data are increasing.
I provide some details on the athlete performance improvement application at https://www.ibm.com/developerworks/community/blogs/jfp/entry/Predicting_Cyclist_Trajectory?lang=en
To your point, this was more about new data than large data.
Glad you find these applications to be impressive.
Big Data is usually defined as the 3: Volume, Velocity, Variety. A 4th V, Voracity, was tacked on later.
Personally, I would have liked the term “New Data” over Big Data since “New” can be argued to cover Velocity, Variety and Voracity. Big is just a synonym for Volume.
There will always be New Data and Big Data. It’s just that today’s New data will be yesterday’s Old data and today’s Big data will be yesterday’s Small data…
In contrast to much discussion regarding “big data”, I found the pair of blog entries on “tiny data” by Rasmus Bååth totally delightful.
See
http://www.sumsar.net/blog/2015/07/tiny-data-and-the-socks-of-karl-broman-the-movie/
and
http://www.sumsar.net/blog/2014/10/tiny-data-and-the-socks-of-karl-broman/
Data are data, but however much there is, it remains, in my opinion, important to question a fifth V, Veracity, more than ever.