Avoiding the AI Money Pit: Truth Bombs, Blind Spots, and Streamlining Storage Receipts

Mention AI, and it’s hard for the conversations not to hit that bottom line: cost. Let's face it, AI doesn't come cheap. But the real issue isn't the sheer number of zeros on the invoice; it's the gaping holes in our understanding of them.

We've got a pretty good grasp on the upfront costs: GPUs, data centers, networking gear — that whole shebang. Even the price of training those neural networks isn't a complete mystery. But once we unleash the AI genie for inference, things get murky.

"It's the hidden costs that are often overlooked — data management, classification, retention policies, compliance, and multi-jurisdictional governance. These can escalate dramatically with AI-generated data,” warns Matthew Oostveen, chief technology officer for Asia Pacific & Japan at Pure Storage.

To increase AI adoption and drive more projects, we must find the right balance between growth and costs. Oostveen believes we need to rethink our approach to budgeting for AI. 

Poking holes in tradition

No one's saying we should ditch traditional financial forecasting entirely. There's still plenty of wisdom in those time-tested practices. But AI throws a few curveballs into the mix, and that's where we need to adapt.

AI projects are notoriously unpredictable, particularly those involving language models (think ChatGPT on steroids). They have a mind of their own when it comes to data demands.

It's the dark matter of AI finance: data management, classification, retention policies, compliance, and the labyrinthine world of multi-jurisdictional governance. These can balloon into budget busters, especially when AI starts generating data like a hyperactive hamster on a caffeine drip.

As Oostveen puts it, “Growth doesn't happen linearly. It happens in stages and steps, often driven by revenue cycles that are notoriously difficult to predict.”

This leads to the all-too-familiar scenario of overspending on storage to avoid bottlenecks or scrambling to add capacity when projects unexpectedly take off. Either way, the bottom line suffers.

So, how can companies navigate this turbulent landscape and ensure their AI ambitions don't sink under the weight of spiraling storage costs? Where is that sweet spot between ambitions and affordability?

It’s a question that Pure Storage tackled head-on during their recent Pure//Accelerate.

Charting a Pure, new course

For Pure Storage, the answer for taming AI costs lies in AI, quipped Oostveen. And he’s not wrong.

Take the company’s new GenAI Copilot for Storage as an example. It's like having a seasoned storage expert on your team, guiding you through the maze of performance issues, management headaches, and security concerns — all while keeping those pesky AI-generated data costs in check.

“We're cutting through the complexity with this tool,” explains Oostveen. “It guides storage teams through intricate performance investigations, management issues, and security concerns, mitigating the cost explosion associated with AI-generated data.”

Another strategy is embracing the “as-a-service” model, which allows companies to scale their storage resources dynamically based on actual usage rather than guesswork.

Data leaders will often roll their eyes when you mention as-a-service. Some even point to the prevalence of this model as creating havoc in IT budgets, primarily when the company uses a CAPEX-heavy approach.

This is where Pure Storage's Evergreen//One, an AI-first storage-as-a-service platform, looks to be different. Yes, it simplifies by bringing cloud-like consumption to data storage, allowing companies to recalibrate their storage needs when models get into the inference stage and need to scale fast and hard.  But more importantly, it provides transparent cost visibility that feeds into your cost model or budget.

“Customers are looking for a simple way to get a handle on their storage costs,” says Oostveen. “Evergreen//One gives them the control and visibility they need to regulate spending and ensure their AI projects don't break the bank.”

Watch out for those blind spots

But taming AI costs go beyond just storage. There are still plenty of pitfalls that can trip up even the most seasoned data engineer.

One major blunder is underestimating the impact of data compliance and regulations on your storage bill. “Centralizing data without a deep understanding of local laws can be a recipe for disaster,” Oostveen warns. “Regulatory surprises can derail AI projects or force costly infrastructure overhauls.”

Take data sovereignty laws today, which are constantly evolving and can be a moving target. If companies don’t have a good grip on these laws or their storage architecture is too rigid to meet them, then costs will mount.

It’s one reason why Pure Storage is pushing its Evergreen//One service. It’s really about customer choice. “So we really do give customers a lot of flexibility and choice here in how they can protect their solutions. And we're continuing to grow that choice for our customers as we continue to add countries to the Evergreen program,” says Oostveen.

Another blind spot is the failure to leverage predictive capacity management tools that can offer valuable insights into future storage needs. For example, Pure Storage's Pure1 platform offers dashboards and predictive analytics that enable companies to optimize their storage investments.

Using a combination of Evergreen//One and Pure1, data leaders can better understand how data costs grow based on key AI projects.

Lastly, companies need to see AI projects in two phases: training and inference. Much focus is currently on the training portion, especially with all the news about AI chipsets. Yet, inference adds a new dimension and requires traditional enterprise IT management know-how.

Acknowledging these two phases and keeping them separate can even allow you to micromanage costs better. “For example, [customers] can say, well, let's keep inference in-house, but we're going to buy pre-trained models,” says Oostveen, who also adds it is an area that gets obfuscated because of the hype. 

Making AI sustainable: Pipe dream and reaching reality?

In this era of eco-consciousness, the sustainability of AI projects isn't just a feel-good buzzword — it's a financial imperative.

Inefficient storage solutions can translate into skyrocketing energy bills, increased cooling costs, and a sprawling data center footprint. Scope 3 just adds to the financial pain and liability.

This is where Oostveen believes Pure Storage has an edge. While many companies focus on data science, Pure Storage is doubling down on its investment in material science.

The result is high-density solutions like the 150TB Direct Flash Modules, due by the end of this year.

Oostveen notes that doubling the capacity isn’t just about cramming more storage into a single piece of hardware. A roomier flash module also reduces data center floor space and power consumption since you do not need twice the power to run a 75TB module.

“Our goal is to ensure that organizations can sustainably grow their AI initiatives without exceeding their original infrastructure footprint,” says Oostveen. “By prioritizing sustainability, we're helping them control costs and achieve their long-term AI goals.”

The winding road ahead

The “AI gold rush” is in full swing, and “there's no shortage of FOMO,” says Oostveen. With many vendors and consultants suggesting that there’s no second-mover advantage with AI adoption, many companies are choosing to jump onto the bandwagon with both their feet.

Oostveen believes pragmatism is slowly dialing down the hype, especially when companies start gawking at their IT bills. What’s crystal clear is that traditional cost control methods are insufficient. When reality hits the market, companies must recalibrate their cost models and ROI picture. 

Pure Storage believes it has the right blend of AI-powered tools and pricing models to guide data engineers and CDOs through this treacherous landscape. This will enable them to harness the full power of AI without sacrificing their financial sanity.

Oostveen concludes, “The key is to embrace change, leverage the latest technologies, and prioritize transparency and control. By doing so, organizations can transform the AI data deluge from a threat into an opportunity.”

Image credit: iStockphoto/jamegaw