Databricks Acquires Okera for AI-centric Data Governance
- By Paul Mah
- May 09, 2023
Databricks has acquired Okera, a startup providing secure data access at scale for data teams, for an undisclosed amount.
This acquisition move comes on the heels of its release of the Dolly 2.0 large-language model (LLM), one of a growing number of open-source AI models that promises to democratize AI for the enterprise.
Databricks says Okera solves data privacy and governance challenges across the spectrum of data and AI. By simplifying data visibility and transparency, Okera helps organizations understand their data better.
Growing importance of governance
According to Databricks, the emergence of AI, especially machine learning models and LLMs, has rendered traditional data governance approaches focusing on narrow layers – such as SQL-centric applications – inadequate.
This is due to the massive growth in data assets that an enterprise must govern and the rapid advancements in AI. Indeed, the latter makes it unfeasible for any single company to establish comprehensive controls for state-of-the-art (SOTA) systems and means that AI-specific governance concerns such as provenance and bias are not addressed.
On its part, Okera’s technology offers an intuitive, AI-powered interface to automatically discover, classify, and tag sensitive data such as personally identifiable information (PII).
Databricks says these tags enable data governance stakeholders to easily assess compliance and create no-code access policies that improve visibility and control over data.
Organizations get access to a self-service portal to quickly audit and analyze sensitive data usage, giving them the ability to reliably monitor and track data usage patterns. Crucially, this ensures governance policies are applied consistently amid the explosive growth of data assets, including those that are AI-generated.
In addition, Okera is testing new governance technology that purports to “support arbitrary workloads” and “without sacrificing performance”.
This intriguing technology is in private preview at the moment, and is currently being tested by some customers on their AI workloads, Databricks says in its press release, promising to share more technical details soon.
“Many organizations don’t have enough technical talent to manage access policies at scale, especially with the explosion of LLMs. What they need is a modern, AI-centric governance solution,” said Nong Li, the co-founder and chief executive officer of Okera.
“We could not be more excited to join the Databricks team and to bring our expertise in building secure, scalable, and simple governance solutions for some of the world’s most forward-thinking enterprises.”
Image credit: iStockphoto/monticelllo
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.