By: Chris D’Agostino, Global Principal Technologist Databricks
The simple answer is no, but there’s a caveat. The failure to pull the needed data insights doesn’t lie with individual employees but the wider organisation. In fact, organisational culture can be one of the biggest barriers to harnessing the full power of data. To ensure everyone across the business is getting the most out of the data available, organisations need to be using the right tools and more importantly, to evolve current attitudes towards handling and sharing data. Otherwise, organisations will continue to fall short when it comes to their data potential.
Simplifying data
Admittedly, however, this doesn’t mean some data knowledge isn’t needed. Nowadays, all workers must understand how to gain insights from data at a general level. But it’s important to remember that there are many different levels of technical expertise needed. Even within a data team, there are various members, from data engineers, data scientists, machine learning engineers to business intelligence analysts.They all work with the data differently and need different levels and kinds of understanding as a result. This same principle applies when widened out to general employees – data doesn’t serve the same purpose for each employee, so the extent to which they need to interact, use and understand it differs too.
As such, general employees need to have access to data but it should be in a format that’s understandable to them, like a dashboard or in a more visual form rather than simply in data tables. This enables them to stay topline or go as deep as they need and to easily pick out what is useful to them. Here, technology is key to ensuring that all employees can better understand and utilise data, no matter their requirements or the level of their data literacy. Data needs to be accessible, easily contextualised and for people within an organisation to be able to interact with it at the level of granularity that meets the needs of their use case(s).
These are the very needs fuelling the likes of the low-code/ no-code movement, for example, with a boom in development platforms that can be used to build apps either completely without code or with just a few lines of use case specific python or SQL code. This requires a new level of collaboration, sharing and code reuse that isn’t available when data is stored in silos and access is controlled on a system by system basis — by a select few individuals. The low-code/no-code movement builds on the upstream work of other data professionals and allows each user to zoom in or out to view the same data in different ways.
It is all about opening up more to users who aren’t necessarily experienced data scientists, to accelerate innovation and boost productivity.
Providing easy access for all
Still, making data more accessible, let alone simplified and visual, poses its problems and this is where organisational culture and policies comes into play. Too often, data can be isolated from many employees and only controlled by a select few – much to the organisation’s detriment. If data is allowed to be shared, it is often fragmented or subsets are created that can cause data drift, further hampering data efforts. At a minimum, all data needs to be in one “logical” place via an unified catalogue approach. Ideally, data is fed from the siloed systems of record to a centralised data store. Having data centralised ensures that data access, machine learning models, and other templates can be configured in an automated way so they can easily be deployed across all departments and business units, allowing for a much healthier data culture.
This is in keeping with the emerging concept of the data mesh, if done correctly. Properly managing the lifecycle of data in a distributed low-cost object store provides the scaling, read/write performance and built-in microservices to allow data encapsulation and access at any level of granularity — supporting all data workloads from a single view of an organisation’s data.
Having the data in one place is just the start, however. As already mentioned, different employees need access to different levels of data granularity. For example, it may be useful for someone in marketing to see a list of customers and when they became a customer, but they do not need to see customer spend. Rather than employees having to pull out the necessary data for themselves or data being fragmented, organisations should be streamlining access to enable end-users to use the data effectively and efficiently to gain the most value from it, and quickly.
Data is everyone’s business
There is no denying that data has become part and parcel of many people’s working lives and some level of data literacy is required, no matter your field of work. Still, re-training or upskilling employees takes a lot of time, money and effort and there are other, better ways organisations can ensure their employees are getting more from data right now. It boils down to using technology and changing approaches to facilitate collaboration, simplify and streamline the presentation of data and most importantly, to grant data access for all employees. We don’t all need to be data scientists but data and AI must be everyone’s business.
