What is Big Data.
The term Big Data is being increasingly used almost everywhere both online and offline. Big Data isn’t new. Forbes traces the origins back to the “information explosion” concept first identified in 1941.
The Big Data is a collection of digital trace (or data) of what we do every day which we (and others) can use and analyse. Big Data therefore refers to that data being collected and our ability to make use of it.
Some experts say that the Big Data Concepts consist of the three Vs: Volume, Velocity and Variety. Below is a further explanation for each.
Volume. A typical PC might have had 10 gigabytes of storage in 2000. Today, Facebook ingests 500 terabytes of new data every day; a Boeing 737 will generate 240 terabytes of flight data during a single flight across the US; the proliferation of smart phones, the data they create and consume; sensors embedded into everyday objects will soon result in billions of new, constantly-updated data feeds containing environmental, location and other information including video.
Velocity. Clickstreams and ad impressions capture user behavior at millions of events per second; high-frequency stock trading algorithms reflect market changes within microseconds; machine to machine processes exchange data between billions of devices; infrastructure and sensors generate massive log data in real-time; on-line gaming systems support millions of concurrent users, each producing multiple inputs per second.
Variety. Big Data isn’t just numbers, dates, and strings. Big Data is also geospatial data, 3D data, audio and video and unstructured text including log files and social media. Traditional database systems were designed to address smaller volumes of structured data, fewer updates or a predictable, consistent data structure. Traditional database systems are also designed to operate on a single server making increased capacity expensive and finite. As applications have evolved to serve large volumes of users, and as application development practices have become agile, the traditional use of the relational database has become a liability for many companies rather than an enabling factor in their business.
Furthermore, Big Data and its implications will affect every single business — from global enterprises to small and medium business— and ultimately change how we do business.
It doesn’t matter what field you operate in or the size of your business; as data collection, analysis and interpretation become more readily accessible, they will have an impact on every business in several important ways.
Data will become an asset to every business.
If a business has a website, a social media presence, accepts credit cards etc., even a one-person shop has data it can collect on its customers, its user experience, web traffic and more. This means companies of all sizes need a strategy for Big Data and a plan of how to collect, use, and protect it. This also means that savvy businesses will start to offer data services to even very small companies. It also means that businesses and industries that never thought Big Data would be “for them” might be scrambling to catch up. For example, if you own or operate a business, and you have questions about how to improve that business, you have data, your data is an asset, and it can be used to improve your business.
Data will enable companies to collect better market and customer intelligence.
It is possible that the companies you do business with already know a lot about you — and the quantity and diversity of what they know about you is increasing every year. Every company (from car manufactures who will monitor our driving to golf ball manufacturers that know how often and how well we play) will get much better insights into what customers want, what they will use, what channels they use to buy and so on.
It is important that companies will need to be proactive about creating and maintaining their privacy policies and all the systems and security needed to protect that user data. As we’ve seen with the recent backlash against Spotify and to a lesser extent Microsoft 10, most people will allow companies to gather this data, but they want transparency around what’s being collected and why and they want the ability to opt-out.
Improving internal efficiency and operations
From using sensors to track machine performance, to optimising delivery routes, to better tracking employee performance and even recruiting top talent, Big Data has the potential to improve internal efficiency and operations for almost any type of business and in many different departments.
Companies can use sensors to track shipments and machine performance, but also employee performance. Companies have started using sensors to track employee movements, stress, health and even who they converse with and the tone of voice they use.
In addition, if data can successfully be used to quantify what makes a good CEO, it can be used to improve the HR and hiring process at any level.
Data is breaking away from just the IT department and becoming an integral part of every aspect in a company.
In the best of all possible worlds Big Data will allow companies to improve the customer experience and their product offering.