The 1% Problem: Why Your Enterprise Data Is Useless to AI

Here’s a statistic that should sober up data and AI leaders: less than 1% of enterprise data is currently being leveraged by generative AI, according to IDC. Not because the technology isn’t ready. Because your data isn’t.

“Traditional manual pipelines often take six to twelve months to build and break whenever source formats change,” says Timothy Lam, solution engineer and AI architect for Data and AI at IBM ASEAN. Six months to prep data that becomes obsolete the moment a vendor changes a file format. It’s the enterprise equivalent of building a bridge that collapses every time it rains.

The core issue is not the obvious but the hidden complexity. According to IDC, 90% of enterprise-generated data is unstructured, including emails, PDFs, contracts, technical manuals, and call transcripts. This isn’t neat rows and columns but tables nested inside hierarchies, semantic relationships buried in prose, and critical context that traditional ETL pipelines simply obliterate.

The lakehouse uprising

IBM’s answer is watsonx.data, what they’re calling a “hybrid, open data lakehouse.” But here’s where it gets interesting: instead of forcing you to choose between data warehouses and data lakes or maintaining expensive duplicates of everything (which is worse), watsonx.data decouples storage from compute entirely.

The architecture stores data in Apache Iceberg format on cheap object storage, then exposes it through multiple engines simultaneously. You spin up Spark for batch processing, fire up Presto for interactive SQL and hit Cassandra or OpenSearch for GenAI vector search.

“Decoupling storage from compute means enterprises can scale compute only when needed and avoid duplicating datasets across lakes and warehouses,” Lam explains. The economic implications are brutal: you’re not paying for idle compute, and you’re not storing seventeen versions of the same dataset across different systems.

However, the real innovation lies not just in the architecture, but in what happens to unstructured data once it enters the system.

RAG has a dirty secret

Retrieval-augmented generation (RAG) was intended to address AI hallucination by grounding models in real-world data. All good in theory; different story in practice. Traditional RAG is noted to be incapable of accurately addressing basic questions. That’s because most RAG implementations treat documents like word salad, chunking text arbitrarily and losing critical context.

IBM watsonx.data’s enhanced RAG pipeline claims 40% better accuracy by doing something deceptively simple: it actually understands document structure. When a PDF enters the system, intelligent document understanding extracts tables, hierarchies, and semantic relations before vectorization. Those embeddings get stored in an integrated vector database inside the lakehouse, keeping retrieval close to governed data.

Then comes the semantic layer, which includes entities, relationships, classifications, and even positional cues, such as table values and calculations. “The system can answer ‘basic’ but detail-sensitive questions with much higher precision,” Lam notes. It’s the difference between an AI that can find a paragraph mentioning “policy 1234” and one that can tell you which specific clause applies to which regional office.

The platform’s semantic automation, facilitated by IBM Knowledge Catalog, goes further, automatically labeling entities, categories, and relationships while creating structured derivatives from unstructured files. That metadata enables semantic filters, such as “region = EMEA,” to work across natural-language queries, thereby improving retrieval precision before the LLM even starts.

The looming governance headache

Here’s the part that keeps data and AI leaders awake in the night: what happens when your AI accidentally leaks PII from a restricted document? IBM watsonx.data inherits source permissions and can annotate sensitive data, so restricted content doesn't make it into retrieval results. “The model isn’t forced to ‘hallucinate’ around blocked context,” Lam says. “That yields fewer wrong answers caused by missing or off-limits passages.”

End-to-end lineage tracks documents from ingestion through AI output, with automatic metadata enrichment capturing tables, hierarchies, and potential PII. Each version gets preserved with lineage information. The platform offers granular role- and attribute-based access controls, encryption at rest and in transit, and certified compliance with SOC 2, ISO 27001, HIPAA, and GDPR.

IBM’s Data Intelligence Assistant further automates governance by suggesting glossary terms and providing intelligent search across datasets. For enterprises where “who accessed what when” isn’t optional, this is a table-stakes requirement.

The agentic endgame

Early results tell the story: some clients have reduced manual data entry by 75% and indexed over seven million documents for AI commentary. But Lam’s vision goes further: “Once enterprises have consolidated and enriched their unstructured data, IBM envisions building fully agentic AI systems that can search, reason and act on the organization’s entire corpus.”

Imagine a domain-specific AI co-pilot that monitors regulatory changes, automatically updates policies and training materials, generates customer-specific compliance documentation, and orchestrates tasks across departments without human intervention. It’s the logical endpoint when you finally unlock that 99% of data is currently gathering digital dust.

The partnership with DataStax, bringing Astra DB’s vector capabilities and the low-code Langflow tool into the ecosystem, addresses the emerging challenges of data variety, duplication, and distributed landscapes that GenAI demands. Petabyte-scale workloads across multiple regions. High availability. Enterprise-grade governance.

Built on over 40 open-source contributions, IBM watsonx.data supports more than 70 file types through 30+ pre-built connectors. The commitment to Apache Iceberg and open standards means no vendor lock-in—your data strategy isn’t held hostage by proprietary formats.

So, with IBM watsonx tighter integration with DataStax, the real question is no longer about whether enterprises can unlock their unstructured data. It’s whether they can afford not to.

Image credit: iStockphoto/Mininyx Doodle