Shijin Pathrose, AVP of Research, opens his mind about how much value the unexplored part of an already available data within an organization aka Dark Data may possess and why organizations need to leverage data governance on dark data.
Though organizations strive to understand and analyze voluminous data to comprehend the dynamic market needs as well as predict human behavior through Big Data Analytics; studies state that more than 50% of big data is dark and not entirely utilized.
Below are his thoughts acknowledging the curious minds on how governing and analyzing this dark data can pose dynamic shifts in innovation and efficiency for organizations.
#1. Why do we need data governance on dark data?
Though termed as dark, the data is prudent, and this dark data has great potential for all organizations, provided there is a structured approach of governance to convert it into smart data which is available for all business purposes. A good governance strategy allows you to have a method in place to validate, cleanse and enrich the data into a usable form that could eventually boost revenues and growth.
#2. Why capturing dark data should be a top priority for an organization?
The dark data is a huge chunk of data that does not have any purpose for the organization unless channelized appropriately. The benefits of laying a governance model and making this data usable are by all means, more cost-effective than managing its storage without a cause. Needless to say, the value of the dark data is not limited and once explored can prove to possess significant value.
#3. What can happen if we don’t govern this dark data?
While Data is imperative, Dark Data is pivotal. Innovation derails when the obvious is ignored. Without proper governance, there is no way to gauge the value of the data, and it would retain valuable and expensive space without allowing us to evaluate the need for retention for business purposes. A good governance model helps in formulating a strategy for retention based on the organization’s business requirement.
#4. What strategies of Data governance do you think are effective on Dark Data?
This varies according to the organization’s prerequisites. A governance model that captures an effective way to identify the data, evaluate the data, project a business need based on cost-benefit analysis, creating of a cost-effective operating model to enhance the quality of data would make it relevant for any organization to invest in dark data.
#5. How this will impact organizations?
Governing dark data will enable us to analyze what is being missed and forecast the future by connecting the missing dots; aiding to comprehend the voluminous unorganized crucial data that is scattered across the organization’s repository. Thus, dark data with data governance best practices not only can incite data re-usability, backup, and analytics but can also aid organizations to build greater insights that may currently be veiled, to accelerate business intelligence.
#6. Data Governance comes with its own challenges; how effective could it be on Dark Data?
Though organizations are skeptical about communication, budget, standardization, and flexibility in terms of data governance today; the future would be a place where data governance is the benchmark to access an organization’s integrity. With the amount of surplus data growing in today’s digital era, certain organizations are already heading towards the possible havoc scenarios that may emerge in case of security and integrity terms and prevising data governance strategies to deal with such situations. Dark Data is no exception.
#7. How much time frame and cost an organization would have to invest in this?
Nevertheless, data governance and dark data is a fledgling couple, with existing and emerging technologies like IoT and AI, the circumstance can be addressed adequately, and the future trillions can be saved; by investing billions today.
#8. How do you think this will shape the future?
Combing the dark data will have a great impact on organizations economically. Apart from business process optimization and innovation, this will also pave way for increased job opportunities in data science, as no sooner organizations plan to establish a stipulated data department headed by Chief Data Officer (CDO).
