Think AI Is Dangerous? Look Closer at Your Data Infrastructure

In the race to implement AI, companies have been fixated on machine learning models, GPU procurement, and crafting the perfect prompts. Critical foundation layers like storage were largely neglected. Now, as companies rush to implement GenAI and eye the benefits of Agentic AI, they are discovering their data infrastructure is woefully unprepared for what’s coming.

“Storage, in all of this, is something of an afterthought,” said Matthew Oostveen, chief technology officer for Asia Pacific and Japan at Pure Storage, during a recent industry panel discussion.

“From my perspective, we are talking to customers who think about storage as [something] they attach to a particular application,” he continued. “It's tuned for that particular use case; then, they kind of forget about it.”

Such oversight isn’t merely a technical inconvenience; with AI taxing current data infrastructures, it’s threatening to become a full-blown strategic crisis.

The AI infrastructure awakening

AI workloads are fundamentally different from traditional enterprise applications. They demand massive throughput for training, lightning-fast access for inference, and increasingly flexible architectures as models evolve at breakneck speed. Even the concept of tiered storage breaks down as models demand access to historic and real-time data for training and inference.

During a Pure Storage Leadership Series panel discussion, “Future-Proofing Storage for AI: A Fool's Errand or a Strategic Imperative?”, panelists highlighted that companies are only now realizing that their storage systems — often designed for an earlier era of computing — may be their biggest AI bottleneck.

“How do we avoid the bottlenecks in the future? How do we ensure that we’re not constructing another castle in the sky that’s going to be problematic when it comes to retaining data, when it comes to sharing information, and when it comes to the ingest side?” Oostveen, who was one of the panelists, asked.

The inherited approach of siloed data storage is proving particularly ill-suited for AI workloads. As Jeanne Pigassou, manager at Wavestone, explained, “What I always took for granted is to have data warehouses that were built for analytics purposes, but indeed it is not the case for everyone.”

This siloed approach creates particular problems for AI implementations, which frequently need to combine data across traditional enterprise boundaries. “If you want to have a holistic view on your customer, you can’t just keep your silos of customer data here and there. You have to have it all in one place,” Pigassou emphasized.

Companies are discovering that their existing infrastructure simply wasn’t designed for the dynamic, high-performance requirements of AI. Data centers optimized for efficiency or traditional workloads now face entirely new demands, resulting in performance bottlenecks and strategic limitations.

The vexing proximity and PoC problems

A major critical factor that is becoming a major concern for AI engineers is data proximity — how close the data is to the computing resources processing it. With AI systems, this proximity isn’t just a nice-to-have; it’s essential for performance. Edge AI and AI models that depend on real-time data demand it.

“You want the two as close together as possible,” Oostveen explained. “And this is where we start to talk about concepts like data gravity. In the past, we made the data do the moving. Now, we’ve flipped [this concept] on its head — data is the new king and the king doesn’t travel.”

This paradigm shift has profound implications for how companies architect their storage systems. Rather than moving vast amounts of data to where the compute happens, companies need to bring their AI processing closer to where the data resides. This reduces latency and can substantially lower costs associated with data movement.

Another well-known issue is about AI proof-of-concepts (PoCs) hitting a wall when they attempt to scale. “What the clients are struggling with, in fact, is to put a dollar value on the advantage they get [with scaling],” said Pigassou. "That’s why there are so few GenAI POCs that turn into production."

Oostveen elaborated, explaining there are "two big reasons” why AI PoCs and pilots fail. The first is around proving that concept at scale. “Yes, it may work in a laboratory. Yes, we can get the LLM to do what we need it to do within the public cloud. But how do we scale that and do that economically?” he noted.

Even when AI pilots demonstrate impressive capabilities, companies frequently discover that their current data infrastructure cannot support the massive computational and storage requirements when deployed at scale. The economics simply don’t work.

“We did some POCs on multi-agent solutions, where you would have queries being, in fact, in natural language information on the databases,” Pigassou recalled. “It worked perfectly. The clients were very happy. And then we did a FinOps view of it, and well, the cost of a query was extremely high.”

Agentic AI: The coming infrastructure storm

If current AI systems are straining data infrastructures, agentic AI — systems that can autonomously act on behalf of users — threatens to break them. These systems, which can perform tasks, make decisions, and take actions with minimal human supervision, introduce unprecedented demands.

For example, agentic AI systems require access to broader data sets, increased processing capabilities, and often need to consolidate information from disparate sources.

The security implications are equally concerning. “What we’re cautious of here is the creation of honey pots from a bad actor perspective,” Oostveen warned. “As we’re starting to consolidate large amounts of data that might be in data warehouses, data lakes, databases... what can we do from a protection perspective to make sure we keep the bad actors away?”

Pigassou agreed, noting that “cybersecurity of AI is definitely something that has to move along with all the improvements. “The hackers and the attackers are also using GenAI and AI to attack. The attacks become more sophisticated, so you have to keep up and have stronger cybersecurity by default and by design.”

Future-proofing data infrastructure is still a strategic imperative

The panel discussion started with a question: Is future-proofing data infrastructure for AI a fool’s errand with its constant evolution or a strategic imperative? The panel’s conclusion was decisive.

“Is it a fool’s errand? No,” asserted Oostveen. “I think it’s an admirable direction of travel. It’s not something that you’re going to solve with the acquisition of a product or a single solution. This is a strategic imperative that incorporates the people that work with the technology, the application stack, the processes, and the infrastructure.”

What makes this particularly challenging is the rapid evolution of AI technology itself. “With every model kind of overachieving the other one a couple of days or one week later... this is definitely the big question where you don’t know what is expected, and you just hope that you won’t have any limitation in accessing these new models that are coming up every minute,” Pigassou observed.

The experts agreed that the key is to build a flexible, adaptable infrastructure that can evolve alongside AI technologies right from the onset. “Avoiding this concept of the siloes and having flexibility in your infrastructure is paramount,” Oostveen emphasized.

His final advice to organizations: “Think strategically. Storage isn’t an afterthought. In the old world it was, but we need to think about how we can access and utilize all the data investments that we have inside our organization, whether in the public cloud, on-premises, or on the edge. Let’s not think in silos anymore. Let’s start thinking holistically at that base layer, and then we can start to solve some of the bigger challenges that we’ve got.”

Those that don’t might just find themselves building impressive AI models on foundations of sand.

Image credit: iStockphoto/LuckyBusiness