The Journey Toward Big Data

Since the 1990’s companies have been seeking to bring together internal and external data sources in order to gain a unique competitive insight other competitors did not have. The main problem was finding these customer insights due to a lack of available external data sources to match against internal data sources.

In the subsequent 20 years, external data sources have exploded based upon companies selling internet-based data, of which social media is one of the key examples. This explosion of data which can be used by companies to create competitive advantage when analysed by data scientists is known as big data.

Big Data Defined

Some have referred to big data as a meaningless marketing tool used to sell hardware and database software. Those who say this have not taken the time to understand what big data is and the fact that big data facilities actually exist.

Specifically, big data can be simply defined as “data compiled from hundreds of sources, most often associated with an industry, available for usage by a company in order to improve the competitive advantage of the company.”

Common Big Data Value Points

Some of the key uses sought from companies’ big data initiatives include:

• Predictive: Big data is often used in order to provide predictive insight regarding how a customer is likely to behave in the future, which would not be known otherwise.

• Defensive: Big data can often be used to predict a customer’s pending adverse activity such as terminating their relationship with the company based upon certain predictive criteria.

• Strategic: Big data can often predict insight to what products and services a company needs to invest in, in order to be more competitive in their future market and acquire more customers.

• Legal compliance: Big data can often be used in order to identify high-risk customer attributes to perform business transactions which would violate legal or compliance regulations.

• Tactical: Big data can often provide insights into a company’s industry and associated trends. This can help the organization modify its business operations in order to save operating costs.

Use Case: IBM

IBM undertook a ground-breaking project with the government of China in order to create a “Smart Electric Grid” where by which the electric utility could receive data over the same infrastructure previously only used for power transmission.

This project allowed for electric utilities to acquire customer consumption data without having to physically read meters. It also allowed for the better planning for expansion of service capabilities as well as detecting suspicious usage patterns that could be investigated in real-time before utilities were potentially forced to absorb unauthorized usage.

The Future of Big Data

Big data initiatives are slowly generating material value largely based upon companies having specific clarity of what they are trying to achieve, along with the necessary data required to facilitate these initiatives.

One key risk associated with big data is the surrounding regulatory environment. While today there might not be specific regulation associated with big data initiatives, companies should not take this lack of regulation as an indication that regulation could still not impact specific company initiatives.

Certainly given the immaturity of big data initiatives companies are far better off outsourcing their initiatives until such time as the technology is proven to add specific business value as well as having clarity that regulatory bodies will not marginalize big data initiative’s.

We live in a real-time world, and the use of big data will certainly be a key tool in ensuring that companies will be able to not only defend market share, but also gain competitive advantage. Big data could be the next phase in achieving these objectives.