The Great Budget Delusion: Maybe We Are Looking At AI Costs All Wrong
- By Winston Thomas
- January 14, 2025

When it comes to budgeting for AI, our dollars and sense may not be perfectly aligned.
It’s no secret that companies have always obsessed over containing costs, especially with markets becoming more uncertain. It makes business sense. Betting on unknowns makes sense when you have a war chest and a strategic need — but it is always done within acceptable risks, especially in large enterprises.
AI is different. Use cases are still forming. Understanding of its impact on business is still evolving. And while many are focused on efficiency and productivity, AI has the potential to revolutionize the market.
So, how should enterprises budget AI? A poll survey at the start of the recent Pure Leadership Series panel discussion revealed that 67% of organizations consider costs their biggest AI challenge, outranking even data infrastructure concerns and governance issues. But the ensuing discussion also noted that this cost-first mindset could be their undoing.
Companies must stop looking at costs and focus on return on investment and value. This fundamental shift in thinking – from cost center to value creator – will separate companies that successfully deploy AI from those stuck in pilot purgatory.
The great AI cost pretension
Matthew Oostveen, chief technology officer for Asia Pacific & Japan at Pure Storage and one of the panelists, observed that 2024 saw something of an “AI arms race” with organizations scrambling to deploy solutions out of fear of being left behind. But this reactive approach is giving way to a more nuanced reality in 2025.
“I expect that we'll see fewer POCs and pilots run for AI, but paradoxically, we'll see an increase in the amount of spend,” Oostveen noted. “The fewer projects we have will be better funded, and they will be largely tested, predicated on the POCs and pilots we've run in the past.”
A crucial tipping point is marked by this evolution. Rather than spreading budgets thin across multiple experimental projects, companies are beginning to concentrate resources on proven use cases with demonstrable returns. It's a long overdue maturation that suggests AI is moving from speculative technology to essential business infrastructure.
But there's another factor that rarely appears on balance sheets: trust. Oostveen cites a revealing statistic from OpenAI: roughly one-third of ChatGPT's outputs in enterprise settings are inaccurate or wrong. “If I'm using that in an enterprise as a tool, and one in every three times it's giving me a false answer, I would rather go down to Happy Valley [Racecourse] because I think I've got better odds of a win,” he said.
This trust deficit has profound implications for AI budgets. Companies are discovering that building reliable AI systems often requires significant investments in data quality, validation systems, and Retrieval Augmented Generation (RAG). While these additions increase upfront costs, they're essential for creating AI systems that business leaders can actually trust with mission-critical decisions.
“You can't easily measure the risk to your organization's reputation,” Oostveen emphasized. “When we're nickel and diming on a little bit of infrastructure and some software and services to get an effective RAG system up and running... we need to step back, take a deep breath and realize that if we invest more wisely, then we're going to get better results.”
Looking to catch up on our recent Pure Leadership Series on AI costs? Rewind and watch the discussion here.
Cloud complexity costs
An unexpected trend in AI infrastructure is the return to on-premises systems. While cloud platforms initially promised cost rationalization and flexibility, many organizations are discovering hidden costs, particularly in data egress fees and long-term storage. However, cloud adoption does accelerate speed-to-market.
Instead, companies need to develop a strategy about what data stays in the cloud versus on-premises, allowing for speed-to-market without blowing a hole in infra budgets. This approach requires constant balancing, especially as data sets grow larger, AI use becomes more prevalent across the enterprise, and regulators rein in data flow.
This hybrid cloud reality suggests that AI budgets must account for a more complex infrastructure landscape than many organizations initially planned for. It is where many got caught out when moving from a project to enterprise-wide deployment. Those who succeed in finding the right balance can adapt as AI workloads evolve and mature.
Oostveen noted that vendors are also realizing that companies need better help. For example, the company’s Evergreen//Forever plans remove some of the budget anxiety and guesswork, so AI teams can focus on moving their models from training to inference and later deployment.
A future of flexibility
As companies move into 2025, they must stop treating AI budgets as a cost-containment exercise and start viewing them as strategic investments in business transformation. This means moving beyond simplistic ROI calculations to consider factors like trust, reliability, and long-term flexibility.
Success requires a fundamental shift in mindset. Rather than asking, “How much will this AI project cost?” companies should ask, “What value can this AI capability unlock?” It's a subtle but crucial difference that could determine how well your company navigates the AI revolution.
Inaction is not an option. AI is poised to reshape your competitors' capabilities. Moreover, unmanaged "Shadow AI" projects, driven by departmental efficiency and revenue needs, are a looming reality. This not only undermines budget visibility and control but also introduces significant data security and privacy risks.
Oostveen offers a simple rule of thumb when it comes to AI costs: think about the future when budgeting for AI. “Think about what systems should look like in two and three years’ time. Think about how you can embed flexibility and choice.” In the rapidly evolving landscape of AI, the ability to adapt may yet be priceless.
Image credit: iStockphoto/beast01
Winston Thomas
Winston Thomas is the editor-in-chief of CDOTrends. He likes to piece together the weird and wondering tech puzzle for readers and identify groundbreaking business models led by tech while waiting for the singularity.