How Tiny Boxes Fuel GenAI Big Dreams (and Save the Planet)

As technologists (or wannabes), we have a hidden talent to simplify our needs until we get burnt.

Generative AI (GenAI) offers a good example. Mention it, and we have voluminous debates about GPU platforms, AI chipsets and the type of datasets on which the large language modules (LLMs) were trained.

However, we often take the infrastructure cogs that keep these data-crunching beasts working for granted. And one major cog—storage—is about to matter much more.

Storage needs an urgent state of consciousness

Linking the dots from GenAI performance (or, for that matter, any AI model) to data is simple.

“But that’s just step one,” says Matthew Oostveen, chief technology officer and vice president for APJ at Pure Storage.

To clean, prepare and transform the data for AI ingestion, we also need the right location to house (or store) these bits and bytes.

Besides performance, your storage needs to be multiprotocol-ready. That’s because you deliver the data to different endpoints or devices.

The importance of storage also goes beyond the training phase. In the inference phase, where we deploy and maintain the model in a production environment, the wrong subsystem can create latency and unnecessary havoc when we extend the data pipeline across the enterprise and geographical lines.

It is also the phase where Oostveen sees many companies stumble.

Back to the (data science) future

Nanyang Technological University did not have the luxury to make mistakes. Its School of Biological Sciences wanted to sequence the genetic makeup of over 1,000 plants they were studying.

Time was in the way. Specifically, the conventional approach would have allowed them to sequence in 20 years, notes Oostveen. A lot of the primary researchers would have already retired by that time.

So, Pure Storage helped NTU deploy its FlashBlade. The improvements were not just about input/output per second (IOPS); they also tightened the entire data pipeline.

This data pipeline is unique for gene sequencing. A DNA sequencer spits out a massive ream of information in small files. The file type needed to be converted to server message block (SMB) files, which an HPC environment works with.

This is where the FlashBlade’s multiprotocol benefit weighed in. “What we did with the FlashBlade was to look at what the existing pipeline was and see what improvements and requirements the HPC environment can be had, and tune a system to keep the GPUs and CPUs at a 100% utilization,” says Oostveen.

There’s also a qualitative aspect to faster storage systems. The speedier data processing allowed the scientists to crunch the results instead of idling away time waiting for the data to be ready.

When GenAI gets physical

While the benefits of GenAI continue to grow, so does the cost of running these models. Just ask Meta.

The IT behemoth chose Pure Storage to house its mammoth AI supercluster. The reason came down to one major differentiator: efficient storage design.

“The reason Meta selected us for this project is not just because of the system's performance—obviously, that was the first box that we ticked. But it was also how we went about storing the data and the kind of footprint we left on their facility,” says Oostveen.

“The supercluster uses a lot of electricity; it takes up a lot of floor space; it emits a lot of heat. And so the vendor selection criteria for Meta are not just around performance but also heavily tread around their facility,” he continues.

While other storage vendors took the apparent route of asking Meta to build another data center, Pure Storage designed and created a unique AI research cluster that made it more “environmentally very efficient” within two years.

“What that means is that Meta does not only have the world’s most powerful AI supercluster but also has arguably one that is most energy efficient,” says Oostveen.

The bonus is the additional energy saved can now be rerouted to spin up more GPUs for other areas of AI research.

“The choice is theirs. The point is that to participate in these kinds of opportunities, you need to be more than a one-trick pony—not only storage, but you need efficient storage,” says Oostveen.

The second coming of storage efficiency

Meta is not the only one. There are plenty of signs that GenAI is creating a newfound appreciation of storage system design. As operating budgets shrink and the business urgency over GenAI explodes, storage efficiency will only matter more.

Pure Storage is taking a pragmatic approach to support this call for efficient storage in driving GenAI.

First, the company invests in AI to rein in AI, specifically by pouring money into AIOps capabilities in its Pure1 offering.

“We are removing the knobs and dials that make storage needlessly complex. We’re ruthlessly simplifying it,” says Oostveen.

Oostveen also ties hardware inefficiency to software deficiency. In his view, the more inefficient your software is, the harder it will drive your hardware.

“Many people do not realize this: we are a software company first and a hardware company second. Instead of looking at the underlying storage as a drive, we communicate directly to flash,” says Oostveen.

Pure Storage is always looking to keep its code base efficient for the hardware drives. “And it is why our ESG scores are so much more efficient than any infrastructure company,” Oostveen adds.

Pure Storage is adding material science wizardry to its software code maintenance. It has enabled it to fit a 75TB DirectFlash Module (DFM) in the palm of a hand.

Creating a small, dense form factor has cooling and power usage advantages. Oostveen also claims their DFMs fail 20 times less than conventional hard drives, which does wonders for data centers' operating budgets.

A lighter form factor also lowers the load on data center floors, adding to operating cost savings—a factor many data center administrators know but business teams seldom appreciate.

The future GenAI-storage war is design

All these factors will matter more as ESG reporting becomes more clinical with incorporating Scope 3 for downstream emissions. Whether you run your data center or use a hyperscaler cloud facility, the emissions are still counted as part of your report.

“You need to consider how best to deploy software in a hyperscaler and whether there are gross inefficiencies that need rectification,” says Oostveen.

With the demands of GenAI coming at the same time as governments turn the screws on ESG reporting to stop greenwashing, it means your storage subsystem will directly impact not just your competitiveness or bottom lines.

The link between GenAI and the storage system has never been more apparent.

Want to know how storage can make or break your GenAI ambitions? Join Pure Storage’s Matthew Oostveen and leading AI practitioners at our upcoming webinar "Will GenAI Break Your Data Infrastructure? An Industry Discussion". Register here. 

Image credit: iStockphoto/ChakisAtelier