Is Your Data Driving Chaos?

One of the mantras about data is that it should be democratized throughout the organization.

Even if they are not data scientists or technology savvy, many business leaders believe that data is like a superpower and that sharing it around and putting it into the hands of people at the coalface is a formula for success.

The reality, however, can often be very different. Democracy might be the goal, but the result is often anarchy.

Bringing order to the data chaos, particularly in the emerging world of AI, was one of the themes addressed by cloud provider Cloudera at the recent data summit held by Gartner in Sydney.

"Data democratization is amazing and wonderful, but when you do that without governance, then you are creating multiple sources of truth," said Shayde Christian, chief data and analytics officer at Cloudera.

“Five instances of the same Salesforce table can wreak unnecessary havoc on computing storage and drive up costs. When you get to the quarterly business review, which should be the most important meeting of the quarter, and if finance has different numbers and metrics from sales and marketing, it's just going to be an argument."

IT in the shadows

Too many organizations, according to Cloudera, are battling the chaos of shadow IT and shadow data that’s manifested by a lack of governance, efficiency, data quality, security and system performance—all accompanied by unexpected cost overruns.

The fast-moving advent of AI adds another dimension of complexity. Still, Christian says that the same principles of data governance and security must apply, and he sees risks among recent adopters who have embraced AI in a rush.

"AI is just another tool to put on top of things to fulfill different use cases; the business case is what should come first,” he said.

"And I see that many people who have adopted earlier are not at so much risk because they have done the governance part and understand the lesson that they shouldn’t allow their IP to leave their property.”

"Data democratization is amazing and wonderful, but when you do that without governance, then you are creating multiple sources of truth."

Christian is not against democracy but says that for it to work, it should have a data management and analytics framework. This gives data users all of the key data, analytics and AI functionality they need in a unified, secure environment.

This requires changes in analytics functionality to be easily configurable and not need additional development and programming in the back end.

Data sources should be captured with flexibility, breadth and support, and there should be support for all types of cloud deployment models because different organizations have different cloud migration strategies.

Another arrow in the quiver

The Cloudera view is that leaders in data democratization use a framework that builds out an enterprise data cloud that is both hybrid, multi-cloud and multi-function, is secure and governed, and supports open-sourced software.

Done right and with good governance, democratizing data from the Edge to AI can have transformative impacts.

In one example, Christian referred to insurance customers using externally sourced images from space to look at events like the California wildfires.

In this use case, AI has been trained on data from damaged homes. The insurers could use this application to approve next-day customer payments without needing site visits.

AI made this possible, but it was implemented within a rigorous, secure and trustworthy framework and began with a starting point, which was the ultimate business goal.

“AI is just another technology, it’s just another arrow in your quiver,” said Christian.

“So it’s really the data we need to protect and secure and be paranoid about, not necessarily the AI tech.”

Lachlan Colquhoun is the Australia and New Zealand correspondent for CDOTrends and the NextGenConnectivity editor. He remains fascinated with how businesses reinvent themselves through digital technology to solve existing issues and change their business models. You can reach him at [email protected].

Image credit: iStockphoto/ronstik