Google adds two new TPU variants for confidential and memory‑intensive AI workloads

Google announced two new TPU configurations—the TPU 8tx for confidential computing and the TPU 8i Max for memory‑intensive workloads—bringing the total number of TPU systems shipped in a single year to four, according to The Decoder.

What happened

The Decoder reports that Google’s eighth‑generation AI accelerator family now includes the TPU 8tx and TPU 8i Max. Both variants use the same silicon as earlier models but differ in packaging, networking, and memory layout. The TPU 8tx is aimed at enterprises and government agencies that need to keep data and model weights encrypted even while the processor is active, a capability Google says matches Nvidia’s confidential computing offering. It relies on Google’s low‑latency Boardfly network and claims up to twice the latency reduction versus the prior TPU 8t. The TPU 8i Max tackles the “memory bottleneck” by placing two TPU 8i chips in a single package, delivering double the compute, memory, and bandwidth in the same rack footprint. Beyond hardware, Google released TorchTPU as open source so PyTorch code can run on TPUs with minimal changes, and it added support for inference frameworks such as vLLM and SGLang. The move reflects a strategic shift from internal‑only use to a broader external platform; the company now counts nearly 100 external TPU customers.

Why it matters for small AI teams

For small companies that run AI‑driven products, the new TPU variants open two practical pathways. First, the TPU 8tx’s confidential‑computing design lets you fine‑tune or distill models on sensitive data without ever decrypting it in memory, reducing compliance risk for regulated industries. Second, the TPU 8i Max’s doubled memory bandwidth can accelerate inference on large codebases, video streams, or multimodal agents that would otherwise stall on standard hardware. Because Google is positioning TPUs as a cloud‑available service rather than a proprietary internal tool, even a modest‑size team can access these accelerators on a pay‑as‑you‑go basis. The open‑source TorchTPU bridge means existing PyTorch pipelines require only minor adjustments, lowering the engineering effort compared to rewriting code for a new framework. In short, the hardware and software updates give small teams a realistic option to boost performance and security without a multi‑year hardware refresh.

What to do this week

1. Identify any confidential data or proprietary models in your current pipelines and run a quick cost‑benefit check to see if TPU 8tx’s encrypted‑in‑memory processing could replace a CPU/GPU step.
2. Clone the TorchTPU repository, run the provided example notebook on a small PyTorch model, and note any code changes needed for TPU compatibility.
3. Reach out to Google Cloud sales for a trial TPU instance (including the 8i Max configuration) and benchmark a memory‑heavy inference task against your existing setup.

Sources

• Google ships four TPU variants in one year as it ramps up competition with Nvidia — The Decoder