● Covered by 6 sources · 8 reportsMedium impact7 neutral1 positive
Google Open-Sources TorchTPU for Native PyTorch on TPUs, Hugging Face Integrates
Key points
- Google open-sourced TorchTPU, a native PyTorch backend for TPUs.
- TorchTPU allows PyTorch models to run directly on Google's TPUs.
- Hugging Face now supports native TPU deployment via TorchTPU.
- This removes the need for PyTorch/XLA or Optimum TPU.
- TPUs are now a native device option for PyTorch, alongside GPUs and CPUs.
- TPUs now serve AI in production at Meta.
- SGLang inference engine has a new PyTorch-native version for TPUs.
- vLLM inference engine also has a new open-source PyTorch-native version for TPUs.
- SGLang-TorchTPU is running in production at Meta.
- RadixArk, a startup, built the SGLang-TorchTPU engine.
- AWS launched TorchNeuron in public beta for AWS Trainium.
- TorchNeuron is an open-source native PyTorch backend.
- TorchTPU was in preview with select customers in April.
- Google has been working on the TorchTPU project since mid-2025.
- TorchTPU was open-sourced at PyTorch Conference North America 2026 in San Jose.
- Meta software engineer Zongwei Zhou stated Meta plans to move larger-scale LLM workloads onto TPUs with TorchTPU.
- PyTorch/XLA was a lazy-tensor bridge from 2020.
- Google Cloud's announcement was co-authored by Bill Jia, VP of engineering for Core ML/AI.
- Bill Jia and Zongwei Zhou presented the keynote "Workload Fungibility in the Agentic Era: Bringing Native PyTorch to Cloud TPUs".
- Mark Collier, executive director of the PyTorch Foundation, stated inference has overtaken training as the largest data center workload.
- Oracle Cloud Infrastructure (OCI) will offer Google TPUs.
- OCI is the first hyperscaler outside Google to offer TPUs.
- TPU deployment on OCI will start in 2027.
- TPU v1 delivered 30x the throughput per watt of contemporary processors.
- Eighth-generation TPUs deliver 12.6 PFLOPS of native FP4 compute per chip.
- Eighth-generation TPUs are over 100x TPU v1's 92 TFLOPS.
- Eighth-generation TPUs offer up to 2x the performance per watt of prior generations.
- TorchTPU supports eager and compiled execution.
- TorchTPU supports distributed computing and hardware-specific optimizations.
- PyTorch/XLA required intrusive source-code modifications.
- PyTorch/XLA suffered from severe compile-time latency.
- PyTorch/XLA lacked support for PyTorch's native imperative execution mode.
- PyTorch/XLA lagged in supporting emerging distributed computing primitives.
- Migrating from GPUs to TPUs with TorchTPU involves replacing the cuda device with tpu.
- Migrating from GPUs to TPUs with TorchTPU involves replacing the nccl distributed backend with tpu_dist.
Native PyTorch on Google TPUs
Google has open-sourced TorchTPU, a native PyTorch backend that allows PyTorch models and operations to run directly on Google's Tensor Processing Units (TPUs). This development was announced at PyTorch Conference 2026 and makes TPUs a native option for PyTorch, alongside GPUs and CPUs.
TorchTPU enables existing PyTorch workloads to run on TPUs with minimal code changes, utilizing familiar APIs. It includes custom ATen kernels for various PyTorch operations and supports integration with `torch.compile()` for optimized execution.
Simplified Development and Deployment
Previously, running PyTorch on TPUs required using PyTorch/XLA (torch_xla), which had complexities such as lazy tensors and graph recompilations. TorchTPU eliminates this, treating TPUs as just another device, simplifying the development and debugging process.
The project provides a compilation cache for optimized execution of XLA computations and includes support for distributed training (Data Parallelism, Tensor Parallelism). Examples for various models like Llama, Qwen, ResNet, DLRM, minGPT, and UNet are available.
Hugging Face Integration
Hugging Face has integrated TorchTPU, allowing direct deployment of models from the Hugging Face Hub to TPUs. This means that models from Transformers, Diffusers, and Kernels can now run natively on TPUs by simply sending them to "tpu" as a device.
This integration removes the need for PyTorch/XLA or Optimum TPU for Hugging Face users, streamlining the process of utilizing Google's AI accelerators for their models.
About Google TPUs
TPUs are Google's specialized AI accelerators, designed for the matrix math operations common in neural networks. Google has been developing TPUs for approximately ten years and uses them to train and serve its own models, including Gemini.
TPUs are available on Google Cloud, ranging from single chips to large pods of thousands of chips, and are intended for high-compute tasks like training and serving AI models at scale.
Broader Ecosystem Support
TorchTPU was co-developed with the PyTorch team at Meta. It supports integration with tools such as vLLM, SGLang, llm-d, TorchTitan, Ray, TorchRec, and LiteRT, providing a consistent software and operational model across TPUs, GPUs, and CPUs.
A user guide and interactive notebooks (Marimo) are available for TorchTPU, which can be run from a TPU VM or a v6e GCE instance.
Updates
- TorchTPU supports eager and compiled execution.
- TorchTPU supports distributed computing and hardware-specific optimizations.
- PyTorch/XLA required intrusive source-code modifications.
- PyTorch/XLA suffered from severe compile-time latency.
- PyTorch/XLA lacked support for PyTorch's native imperative execution mode.
- PyTorch/XLA lagged in supporting emerging distributed computing primitives.
- Migrating from GPUs to TPUs with TorchTPU involves replacing the cuda device with tpu.
- Migrating from GPUs to TPUs with TorchTPU involves replacing the nccl distributed backend with tpu_dist.
- Oracle Cloud Infrastructure (OCI) will offer Google TPUs.
- OCI is the first hyperscaler outside Google to offer TPUs.
- TPU deployment on OCI will start in 2027.
- TPU v1 delivered 30x the throughput per watt of contemporary processors.
- Eighth-generation TPUs deliver 12.6 PFLOPS of native FP4 compute per chip.
- Eighth-generation TPUs are over 100x TPU v1's 92 TFLOPS.
- Eighth-generation TPUs offer up to 2x the performance per watt of prior generations.
- TorchTPU was open-sourced at PyTorch Conference North America 2026 in San Jose.
- Meta software engineer Zongwei Zhou stated Meta plans to move larger-scale LLM workloads onto TPUs with TorchTPU.
- PyTorch/XLA was a lazy-tensor bridge from 2020.
- Google Cloud's announcement was co-authored by Bill Jia, VP of engineering for Core ML/AI.
- Bill Jia and Zongwei Zhou presented the keynote "Workload Fungibility in the Agentic Era: Bringing Native PyTorch to Cloud TPUs".
- Mark Collier, executive director of the PyTorch Foundation, stated inference has overtaken training as the largest data center workload.
- AWS launched TorchNeuron in public beta for AWS Trainium.
- TorchNeuron is an open-source native PyTorch backend.
- TorchTPU was in preview with select customers in April.
- Google has been working on the TorchTPU project since mid-2025.
- TPUs now serve AI in production at Meta.
- SGLang inference engine has a new PyTorch-native version for TPUs.
- vLLM inference engine also has a new open-source PyTorch-native version for TPUs.
- SGLang-TorchTPU is running in production at Meta.
- RadixArk, a startup, built the SGLang-TorchTPU engine.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub huggingface/blog GitHub google-pytorch/torch_tpu GitHub openxla/tokamax GitHub vllm-project/vllm-torchtpu GitHub sgl-project/sglang-torchtpu GitHub google-pytorch/torchtitanHow outlets covered it
Google and Meta have open-sourced TorchTPU, a new project providing native PyTorch backend support for Google TPUs. This allows PyTorch workloads to run on TPUs with minimal code changes, supporting eager execution and distributed computing, addressing limitations of the previous PyTorch/XLA approach.
Oracle and Google Cloud are partnering to bring Google Tensor Processing Units (TPUs) to Oracle Cloud Infrastructure (OCI), making OCI the first hyperscaler outside Google to offer TPUs. This collaboration provides OCI customers with access to Google's custom AI accelerators for demanding AI workloads, with deployment expected to begin in 2027.
Google and Meta open-sourced TorchTPU, a native PyTorch backend for Google's Tensor Processing Units (TPUs), at PyTorch Conference North America 2026. This development allows PyTorch models to run natively on TPUs, replacing the older PyTorch/XLA bridge which required intrusive code modifications. The new backend simplifies model porting and aims to support large-scale LLM workloads on TPUs.
Google open-sourced TorchTPU for its TPUs and AWS launched TorchNeuron in public beta for its Trainium chips, both providing native PyTorch support. These releases aim to simplify running PyTorch workloads on their respective AI hardware, offering alternatives to Nvidia GPUs.
Google's custom AI chips, TPUs, are now running AI workloads in production at Meta through a new PyTorch-native version of the SGLang inference engine. This development allows Meta to utilize TPUs without migrating to Google's JAX AI framework, providing an alternative hardware architecture for its PyTorch workloads.
Google has open-sourced TorchTPU, a native PyTorch backend that allows PyTorch models and operations to run on Tensor Processing Units (TPUs). This development provides PyTorch users with direct access to Google's specialized AI accelerators, potentially improving performance for machine learning workloads.
Google has open-sourced TorchTPU, enabling native PyTorch workloads to run on Google's Tensor Processing Units (TPUs). This development allows PyTorch users to deploy models on TPUs with minimal code changes, integrating TPUs as a native option alongside GPUs and CPUs for AI model training and deployment.
Hugging Face now supports native PyTorch on Google's Tensor Processing Units (TPUs) via TorchTPU, allowing direct deployment of models from the Hugging Face Hub to TPUs. This integration removes the need for PyTorch/XLA or Optimum TPU, simplifying development and deployment for AI models on Google Cloud's specialized hardware.