Challenges and Opportunities in Big data Analysis
Govciooutlook

Challenges and Opportunities in Big data Analysis

By Gov CIO Outlook | Tuesday, November 20, 2018

Researchers and vendors in the APAC region have recognized a number of benefits in adopting big data analytics into business practices. Survey reports portray that big data investments in the automotive industry will account for more than $3.3 Billion in 2018 alone. Over the next three years, these investments are further expected to grow at a CAGR of 16 percent approximately. In today’s global economy, to maintain a competitive edge, all enterprises should understand the need for analyzing data. Along with different types of data, a massive quantity of data is being accumulated every fraction of second and requires organizations to make real-time decisions and responses. But existing analytical tools are not capable of extracting useful information in real time from huge volumes of data.     

The main problem in the analysis of big data is the lack of coordination between database systems and analysis tools (such as data mining and statistical analysis). According to the NewVantage Partners survey, the most common goals associated with big data projects include establishing a data-driven culture and accelerating the speed with which new capabilities and services are deployed. These can be achieved only if they can extract insights from their big data and act quickly. To retain this speed, organizations should look for a new generation of ETL and analytics tools that reduce reports generating time dramatically.

The variables associated with big data leads to challenges in integration of data. As the big data comes from different streams like enterprise applications, social media, and email systems, it’s hard to combine all the data and reconcile it. In an IDG report, 89 percent of the participated crowd surveyed, shared that they intended to invest in new big data tools in the next 12 to 18 months.

However, many organizations seem to believe that their current data security methods are sufficient for their big data needs. In the survey, 39 percent said that they were using additional security software for their big data analysis. Among those who do use additional steps, the most popular ones include identity and access control, data segregation, and data encryption.

Weekly Brief

New Editions