Home Business Solving the growing enterprise data dilemma

Solving the growing enterprise data dilemma

by wrich

By Danny Sandwell, data governance strategist at Quest 

Danny Sandwell, data governance strategist at Quest

Organisations broadly agree that their data can be a differentiator and its effective use is existentially important. In fact, according to ESG’s 2021 State of Data Governance and Empowerment Insights report, 84% of organisations agree that data represents the best opportunity to develop a competitive advantage during the next 12-24 months. These organisations recognise that if they do not continue to find new ways to use data to proactively customise products/offers for customers, they will likely be disrupted by competitors that figure it out before them. 

Businesses constantly collect and analyse data to make informed strategic and tactical decisions – such as improving products, enhancing customer services and reducing expenses. But many organisations are still failing to extract the full value that comes from utilising all data assets. 

So how can businesses become more adept at extracting the full value data has to offer, and just how critical is this? 

The first piece of the puzzle 

The first step to solving this dilemma is to become truly data-driven, yet few organisations have achieved this. Many are so overwhelmed by the data they are flooded with that they are not able to reach deeper conclusions about how to drive revenue, achieve regulatory compliance or make other strategic decisions.

IDC reports 95% of organisations integrating up to six different types of data across 10 different types of data management technologies. The move to the cloud has also been accelerated as secure access to data and computing resources is required from anywhere work is being done. In 2019, 94% of organisations were integrating data across hybrid cloud environments according to the results of the survey referenced previously.

The diversity, distribution, and scale of data in the digital economy are challenging data enablement in the organisation. Effects of data environment complexity and the state of intelligence about data are being seen in the efficiency and effectiveness of data-native workers. The 80/20 rule stating the percentage of time spent in data discovery and preparation compared with the percentage of time spent in analytics is getting worse, now approaching 85/15 as per the results of IDC’s 2019 DII survey. This survey also told us that on average, data-native workers are more unsuccessful than successful in their tasks as they search for, prepare, govern, and analyse data. 

For some businesses, they simply do not know exactly what data they have and often have no idea where some of it is located. They struggle to integrate known data from numerous systems in various formats – and this only becomes more acute where organisations do not have a way to automate those processes. If businesses do not have a grasp of the data they have, they simply cannot use it to its true potential. 

In addition, this is not a problem that is going to go away anytime soon. Recent research indicates that this is in fact actually growing. The collective sum of the world’s data is projected to reach 175 zettabytes by 2025 (and one zettabyte contains a billion terrabytes) – according to the Data Age 2025 report by IDC. The research also revealed that enterprises around the world have the unenviable task of managing more than 97 percent of the global datasphere. 

So, with vast volumes of data, and difficulty when it comes to having visibility, it hardly comes as a surprise that businesses are facing a dilemma and are unable to maximise data to its true value. 

In addition, without a clear picture of all the data within the business, this is going to hamper the ability to reach a single version of data truth. Leveraging snippets of data or simply missing complete datasets is going to prevent organisations from having accurate insights when it comes to the bigger picture. 

The good news is that businesses can become adept at wringing significant value from data, and a large part of the answer lies in harmonising data management with data governance. So, let’s explore what both of these really are and what makes a good strategy. 

Data Management and Data Governance 

Data management drives the design, deployment and operation of systems that deliver data assets for analytics purposes. Data governance delivers these data assets within a business context – by tracking their physical existence and lineage and maximising their security, quality and value. 

Combined they form a critical hub for data preparation, modelling and governance, enabling businesses to solve the data challenge and extract true value. 

Simply put, data management involves knowing what data the organisation has and also where it lives. Businesses need to create and sustain an enterprise-wide view as well as easy access to underlying metadata in order to achieve this. 

Businesses need a real-time, accurate picture of their data landscape and many organisations are actively striving to achieve this. This includes insights across all levels, from data at rest in databases, data warehouses and data lakes, through to data in motion. While this may seem like a tall order, with various datasets that were never designed to work together, it is a critical step to building a comprehensive data management strategy. 

However, there is technology widely available helping businesses to achieve and automate this – from encompassing data cataloguing, mapping, versioning, business rules, glossaries maintenance and metadata management. 

Data governance on the other hand is about being able to pinpoint what data exists and where, but this is also combined with a business’ understanding of what it all means and where it is adopted across the enterprise. Having that consistency is the only way to assure that insights generated by analysts are both useful as well as actionable, regardless of the business department or the user exploring a question.

Policies, processes and tools that define and control access to data by roles and across workflows are also critical for security purposes. These issues can be addressed with a comprehensive data governance strategy and technology to determine master datasets, discover the impact of potential glossary changes across the enterprise, audit and score adherence to rules, discover risks, and apply security to data flows in a cost-effective way.

The result of all this can be an automated, real-time, high-quality data pipeline being established for all stakeholders, serving as a key element for standing up data governance in agile, efficient and cost-effective ways. Everyone from data scientists, data stewards, ETL developers, enterprise architects, business analysts, and compliance officers to chief data officers and chief executives can access the data they are authorised to use and base strategic decisions on what is now a full inventory of reliable information.

Gaining actionable insights 

Combined, data management and governance approach data from both an IT driven and business-oriented viewpoint, leaving organisations well positioned to achieve the main goal we have all been striving for – extracting the true value from data. While this concept is still new for many organisations, the advantages are clear. 

In addition to promoting data-driven insights and business transformation by knowing what data exists within the business, where it is and its value potential, the integration of data management and governance capabilities supports industry needs to fulfil regulatory and compliance mandates.

With this approach, every stakeholder can accelerate the transformation of data into accurate, actionable insights – innovating and transforming the businesses to stay ahead of competitors in today’s fast-paced world. Enterprises can finally solve the dilemma of data and make it work for the businesses at every level, by leveraging all their data assets to their true potential. 

You may also like