Google’s Newest TPUs Have No New Chips. That’s the Point.
- By Winston Thomas
- November 15, 2026

Near the end of a Google Cloud media briefing, someone asked a security question. Google's TPU Sync software moves an AI agent's working memory off the chip when the agent pauses. Would that data stay encrypted on TPU 8tx, Google's first TPU built for confidential computing?
Mark Lohmeyer, vice president and general manager of AI and computing infrastructure at Google Cloud, was honest. “I’d have to check,” he said. “I don’t actually know that.”
Jason Monden, a group product manager at Google, had the answer. The data travels over encrypted PCIe links, the high-speed connections between the host processor and the AI chip. “It remains fully encrypted,” he said.
That is not a gotcha. But it shows where the product now lives. The answer was not in the chip. It was in the links, processors and software around it.
That is the real story behind Google's Nov. 9, 2026, launch of TPU 8tx and TPU 8i Max. Neither uses new silicon. The 8tx runs on the same chip as the TPU 8t, Google's training system. The 8i Max puts two TPU 8i chips, built for inference, in one package. Google draws the line itself: a new chip takes years, while a new system, the compute, networking and storage around the chip, can arrive in months. For a chief AI officer buying compute for agents, and a chief data officer deciding what those agents may see, that shift changes the questions.
Lohmeyer oversees both Google’s TPUs and the Nvidia GPUs it rents out, and “customer choice” is his pitch. Every performance figure below is Google’s own. Google has published no prices. Amin Vahdat, its chief technologist for AI and infrastructure, points to 2027 for shipments.
Read past the specs, and Google’s case rests on four claims: specialized hardware, encryption during use, more memory and portable software. Each holds up, in part.
Specialization is the new speed
TPUs, or tensor processing units, are Google’s custom AI chips. Google once shipped one TPU system every two years, Lohmeyer said, then one a year. In April 2026, it introduced two: the 8t for training and the 8i for inference, the work of running a trained model. Google claims the 8t delivers up to 2.7 times the training performance of the previous generation, and the 8i 80% better inference price-performance. The November announcements make four systems in one year.
Lohmeyer pinned the pace on agents. An agent, as he described it, breaks a request into steps, calls models and tools, and writes and runs code. “What might have been a simple single model interaction can become tens or hundreds or thousands of operations happening behind the scenes,” he said.
“It even protects it from people that are operating the system.” — Mark Lohmeyer @ Google Cloud
Independent forecasts point the same way. Deloitte projects that inference will account for about two-thirds of all AI compute in 2026, up from a third in 2023.
For CAIOs, specialization cuts both ways. Lohmeyer promises “great performance at the lowest possible cost.” But one buying decision becomes four, before you add the Nvidia systems Google also rents, including A5X instances built on Nvidia’s Vera Rubin platform.
The first of the two new systems takes on the question that opened this story: security.
Encryption moves inside the accelerator
Data is usually encrypted when it is stored and when it is sent, Google notes. But it must be decrypted in memory while a computer works on it. That gap is a risk when the data includes model weights, the trained parameters that give a model its value, or customer records inside prompts.
TPU 8tx is “the first TPU that features confidential computing capabilities,” Lohmeyer said. Confidential computing keeps data encrypted and isolated even while it is processed. Google pairs 8t silicon with AMD x86 host processors. Data stays encrypted across the network links, along the path from the CPU to the TPU, and inside the TPU chip and its memory, Lohmeyer said. Hardware attestation, a cryptographic check that the system is what it claims to be, covers the whole system.
“This protects the model weights, it protects the prompts, protects the enterprise data,” he said. “It even protects it from people that are operating the system.”
This level of protection matters most outside Google’s data centers. Google will let select customers run 8tx in their own facilities. There, Lohmeyer said, the data stays protected even from the people who operate the facility. He named two target buyers: highly regulated industries and model providers that want to protect their weights "in a wide variety of environments."
Three caveats apply. First, the category is not new. Nvidia already sells confidential computing on its GPUs and says its Blackwell chips run it at about 98% of normal performance. The 8tx brings the feature to TPUs.
Second, the speed claim needs a careful read. Google says 8tx offers “up to 2x lower latency,” but it compares 8tx with the 8t, a training system. It does not compare 8tx with the 8i, whose low-latency Boardfly network, a design that shortens the paths between chips, 8tx borrows.
Third, scope. Google’s announcement describes confidential computing for 8tx only. It does not say whether 8i Max has it.
That second system targets a different limit: memory.

Memory is the new meter
Chip engineers call it the memory wall. Processors stall because memory cannot feed them data fast enough. When dozens of agents work across a codebase or hours of video at once, Google says, memory bandwidth becomes the bottleneck. More memory supports larger context windows, the material a model can hold in view at once, and larger KV caches, the model’s working notes on what it has read, Lohmeyer said.
Without more memory, he said, companies face three bad options: shrink the context and lose accuracy, run fewer workloads at once or accept slower responses.
TPU 8i Max doubles compute, doubles on-chip SRAM (fast memory built into the chip) to 768 MB, and doubles memory bandwidth to 17.2 terabytes per second, Lohmeyer said. High-bandwidth memory (HBM), the larger memory stacked beside the chip, rises to 384 GB, a third more than the 8i’s 288 GB.
“We’re basically adding more lanes to the highways,” he said. “Not necessarily making each car on the highway go faster.”
Asked how much more efficient 8i Max is, Lohmeyer called it “2x everything,” adding that this was “probably the simplest way to say it.” His own numbers show the simplification: memory capacity rose by a third, not double. The Next Platform calculates that each chip in the package carries 192 GB, down from 288 GB in a standalone 8i. That leaves less memory per unit of compute.
For a CDO, what gets into this memory matters. Lohmeyer described that agents pull into context "files, documents, emails." This is a data inventory and if you need more memory for more context, it becomes a cost decision as well as a governance one.
Faster chips and more memory help only if teams can move their software onto them. That is where Google spent most of its pitch.
Portability moves the lock-in up the stack
TPUs were long tied to Google’s own TensorFlow and JAX frameworks. In October 2026, Google open-sourced TorchTPU, built with Meta’s PyTorch team. It lets code written in PyTorch, which Lohmeyer called “far and away the most popular framework” for AI, run natively on TPUs. Developers “literally just change a config line or two,” he said.
Meta already runs production work on TPUs this way. “We cannot be constrained by a single hardware architecture,” said Jana van Greunen, Meta’s director of PyTorch engineering.
Google can also mix hardware within a single workload. Lohmeyer said Google Kubernetes Engine (GKE), Google's managed service for running containerized software, can start a job on Nvidia-based A5X instances and send the overflow to TPU 8i when capacity runs out. Google's TPU Sync software, which it says cuts time to first token (the wait before a model starts answering) by more than 90%, also runs on older TPUs, back to the fifth generation.
“We’re basically adding more lanes to the highways. Not necessarily making each car on the highway go faster.” — Mark Lohmeyer @ Google Cloud
For CAIOs worried about lock-in to CUDA, Nvidia’s software platform, this is real progress. Lohmeyer stressed openness: the open-source inference engines vLLM and SGLang now run on TPUs, and GKE's inference layer uses the open-source llm-d. Still, in his own example, the tool that moves work between GPU and TPU is GKE, a Google service, so the lock-in does not disappear. Instead, it moves from the chip to the orchestration layer. Oracle plans to offer TPUs from 2027, which gives buyers a second provider.
One detail needs clarifying. In the briefing, Lohmeyer described TPU Sync as the tool that moves a paused agent’s memory off the chip. Google’s blog gives that job to Agent Compute Engine, a separate service, and describes TPU Sync as a layer that moves KV caches between chip memory, host memory and storage. The two may handle related tasks at different levels. Buyers should ask which product does what before they sign.
Those open questions shape what buyers should do before the hardware ships.
Five questions before 2027
Price the task, not the chip-hour: Ask for the cost of one completed agent task at your real context length. The Next Platform expects 8tx to carry a slight premium and 8i Max to cost a little under twice an 8i per hour. Treat those figures as estimates until Google publishes prices.
Bring your own benchmark: Google’s results on the InferenceX and AgentX benchmarks ran on its own software. Test with your own agents and documents.
Put attestation in front of your auditors: Ask what evidence 8tx attestation produces, who checks it and whether your regulator accepts it. Check data residency separately. Confidential computing controls who can see data, not where it is stored.
Map security to each system: Ask Google whether 8i Max supports confidential computing. If it does not, your longest-context work and your most sensitive work may need different hardware.
Test the exit: Run the same PyTorch model on GPUs and TPUs, then move it back. If the move works only inside GKE, put a price on that dependency.
The chip comes with a spec sheet. The system needs due diligence.
Image credit: iStockphoto/Jackie Niam; Chip photo: Google
Winston Thomas
Winston Thomas is the editor-in-chief of CDOTrends. He likes to piece together the weird and wondering tech puzzle for readers and identify groundbreaking business models led by tech while waiting for the singularity.