In a dynamic, global economy, organizations have begun to more heavily rely on insights from their customers, internal processes and business operations in order to uncover new opportunities for growth. In the process of discovering and determining these insights, large complex sets of data are generated that then must be managed, analyzed and manipulated by skilled professionals. The compilation of this large collection of data is collectively known as “big data".
Most professionals in the industry consider multiple terabytes or petabytes to be the current big data benchmark. Others, however, are hesitant to commit to a specific quantity, as the rapid pace of technological development may render today’s concept of “big” as tomorrow’s “normal.” Still others will define big data relative to its context. In other words, big data is a subjective label attached to situations in which human and technical infrastructures are unable to keep pace with a company’s data needs.
V’s of Big Data
Though the word “big” implies such, big data isn’t simply defined by volume, it’s about complexity. Many small datasets that are considered big data do not consume much physical space but are particularly complex in nature. At the same time, large datasets that require significant physical space may not be complex enough to be considered big data.
In addition to volume, the big data label also includes data variety and velocity making up the three V’s of big data – volume, variety and velocity. Variety references the different types of structured and unstructured data that organizations can collect, such as transaction-level data, video and audio, or text and log files. Velocity is an indication of how quickly the data can be made available for analysis.
In addition to the three V’s, some add a fourth to the big data definition. Veracity is an indication of data integrity and the ability for an organization to trust the data and be able to confidently use it to make crucial decisions.
Uses of Big Data
As stated earlier, organizations are increasingly turning to big data to discover new ways to improve decision-making, opportunities and overall performance. For example, big data can be harnessed to address the challenges that arise when information that is dispersed across several different systems that are not interconnected by a central system. By aggregating data across systems, big data can help improve decision-making capability. It also can augment data warehouse solutions by serving as a buffer to process new data for inclusion in the data warehouse or to remove infrequently accessed or aged data.
Big data can lead to improvements in overall operations by giving organizations greater visibility into operational issues. Operational insights might depend on machine data, which can include anything from computers to sensors or meters to GPS devices. Big data provides unprecedented insight on customers’ decision-making processes by allowing companies to track and analyze shopping patterns, recommendations, purchasing behavior and other drivers that are known to influence sales.