The use of big data is no longer a strategy for advanced and technologically innovative companies to gain a competitive edge. It is a necessity for all businesses across all industries. Collecting data is not enough; businesses need to understand how to translate their data into competitive strategies and advantages. It is only then that big data can help a business to have a competitive edge over the competition.
The Growth of Big Data
There are billions of internet users around the world and the amount of data they create is colossal. This year alone, global IP traffic is expected to reach 1.1 zettabytes. This translates to 1,100 billion gigabytes of data. On a monthly basis, IP traffic is expected to reach 88,700 million gigabytes this year. In 2015, per capita monthly IP traffic stood at 10GB. On the other hand, the number of networked devices per capita stood at 2.2. In total, there were 16.3 billion networked devices in 2015. This shows that we are living in an increasingly interconnected world. Between 2010 and 2015, the number of internet users around the world grew by 75 percent to 3.5 billion. It is interesting to note that approximately 40,000 Google searches were executed per second in 2016. This translates to 3.5 billion searches per day and 1.2 trillion searches per year throughout the world. This is a 20 percent increase from the 998 billion searches executed in 2010. When it comes to social media, Facebook had an average of 1.18 billion daily active users around the world in September 2016, and Instagram had over 600 million users.
Growth of Data Usage Within Businesses
The average business enterprise manages 347.56TB of data, which is seven times greater than the amount of data managed by the average small business (47.81TB). On the other hand, the average company manages 162.9TB of data and the volume is expected to increase to 247.1TB of data in the next 12 to 18 months. This is an expected increase of 52 percent.
Big data is an important investment for the vast majority of business executives. According to recent studies, 59 percent of global executives believe that data and data analysis are vital to their business. Thirty percent of business executives believe that big data is very relevant to their business. Nine percent believe it is somewhat relevant while 3 percent of business executives believe that big data is only marginally relevant. According to the study, there is no business executive who dismissed big data as irrelevant to their business. It is interesting to note that 83 percent of companies report employing data to increase profitability of existing products or services. On the other hand, 69 percent of companies would support a new business unit that is dedicated to data-related products and services. In 2016, 48 percent of companies had invested in big data while 26 percent of companies plan to invest in big data within the next two years. Twenty-seven percent of Fortune 1000 firms plan to invest at least $50 million in big-data initiatives in 2017. Another 20 percent plan to invest up to $10 million. The key drivers of big-data investments differ from one firm to the next. For instance, better business insights is the driving force behind 37 percent of businesses that are planning to invest in big data. For 17 percent and 13 percent of businesses, faster decisions and faster speed-to-market are the respective key drivers. Lastly, the key driving force behind big-data investments for 9 percent of firms is stronger analytical capability attributed to big data, while the need to develop a data-driven organizational culture was the key driver for another 9 percent of firms.
Challenges of Managing Big Data
There are many challenges facing big-data initiatives. The main challenge, however, is data pollution. A whopping 87 percent of companies report that bad data pollutes their data stores. Some of the data errors include: incomplete or missing data (60 percent), outdated information (54 percent), duplicated information (51 percent), inconsistent data (37 percent) and typos (30 percent). The biggest cause of these inaccuracies is human error (56 percent). Other causes of big data errors include: lack of internal resources (31 percent), inadequate data strategy (28 percent), inadequacies in current relevant technologies (27 percent) and lack of relevant technology (26 percent).
It is interesting to note that 94 percent of companies have encountered internal challenges while trying to improve data quality. The top internal challenges include: lack of budget, lack of knowledge or skills, justification of investment, lack of qualified staff, auditing/measuring of progress and success, time-to-value expectations, business-data integration, lack of executive sponsorship for long-term data strategy, lack of board buy-in, and under-performing—or lack of—data-quality tools.
External Data Challenges
Businesses also experience external challenges to managing their data. These include: data capture and validation (46 percent), security and governance risks (39 percent), data profiling and discovery (34 percent), managing the volume of data (25 percent), and managing the diversity of data (24 percent).
Business Practices for Getting the Most Out of Your Data
Have a Well-Developed Data-Management Strategy: Companies with strong data-management strategies normally get the best results. Research has shown that 54 percent of companies that rate their data strategy as vital report a competitive advantage over their peers. In comparison, only 26 percent of companies with a less significant focus on strategy report such an advantage. Engage Executive Leadership and Sponsorship: Having a strong business leadership was cited as a critical factor for big-data adoption and success by 23 percent of firms. This is the No. 2 factor for adoption of big-data initiatives. Aim to Solve a Specific Business Problem: Big-data strategies for security and compliance will differ greatly from those designed to increase customer loyalty. Addressing specific business problems yields measurable results. Consider How Data Will Be Used Across Different Business Units: The more widely big data is used across a business, the more likely that the business is going to be profitable. Businesses with profitable initiatives report that big data is 72 percent operationalized across their enterprise compared to just 49 percent of those that have yet to achieve profitability. Put a Priority on Hiring Skilled Staff: Approximately 59 percent of US firms expect the number of jobs that require data-analysis skills to increase in the next five years. In the last 12 months, 78 percent of US organizations reported difficulty recruiting for data-analysis positions. Develop a Secure and Strong Relationship Between Business and Technology: This is the No. 1 most crucial factor. Thirty-four percent of executives rate a solid business and technology partnership as a critical factor for adoption of big-data initiatives. Clean Up Your Data as it Comes from the Source: According to big-data professionals, sanitizing data from the source is the most important quality practice for professionals. Embrace the Variety of Tools, Technology and Platforms: Fifty-nine percent of professionals report that there is a significant operational advantage to upgrading big-data components.
Source: Villanova University