The ability to effectively analyze the data at a company’s disposal has always been a crucial component of any business, but with the exponential growth of the Internet in recent years these pools of information have swelled to veritable oceans of data, with waves of new information continually crashing on shore with the regularity of the tides.
For companies — both large and small — managing the seemingly-overwhelming volume of information and being able to turn it into something meaningful becomes a daunting task. And while the challenge is intimidating, failure to glean the insights provided by these massive data sets is simply not an option for companies hoping to compete in the 21st century digital marketplace.
Many thought leaders in the business sector believe that understanding and adapting to the big data movement has already become factor that separates the wheat from the chaff in the enterprise world; those that can navigate the challenges inherent in big data management will have a significant advantage in the realm of innovation and productivity growth versus those that are unwilling and unable to dive into the big data waters.
Let’s take a look at a few big data factoids that can give newcomers to the concept a better idea of just what we mean when we say “big data”:
- Recent innovations have caused the amount of data to explode well beyond our capacity to manage it. It is estimated that 90% of the digital data in the world has been created in the last two years alone.
- It is estimated that by 2015, the amount of data in the world will be twice the amount that we currently possess. It will double again by 2017, at the latest.
- As of today, it is estimated that we have created three quintillion bytes of data (a quintillion has 18 zeros!). If you put all that data onto standard CD-roms and stacked them, they would stretch to the moon … and back!
- As an indicator of who is creating these massive amounts of data and how, consider this: Every day 12 terabytes (a trillion bytes) of data are created each day on Twitter alone.
- Further indication of how this data piles up through the everyday activities of your average net user: every minute 571 new websites are created, 48 hours of video is uploaded to YouTube and 700,000 posts are created on Facebook.
- Of course, big data is much more than just websites and social media; every credit card transaction, government data entry and cell phone call adds to the perpetual growth of data pools.
- The big data industry has grown from an afterthought ten years ago to a $500 billion industry by 2012.
- It is estimated that this is only the tip of the big data iceberg; in fact projections indicate that there will be a massive shortage of big data analysts in the United States in the next few years. The supply of qualified individuals will fail to reach the demand for their services by as many as 500,000 persons, according to estimates.
- The demand will be so high partly because so much money is riding on big data. By 2015, $16.9 billion dollars will be spent on big data, and those that are able to mine the information effectively are estimated to see a 60% increase in profits!
Richard Turkel writes about all aspects of business technology, database management to IT leadership. He currently writes for BMC, a company that offers IT finance software.