On Tuesday, Google announced that TorchTPU, its long-awaited version of PyTorch, is now available as an open source project. With this, Google joins Amazon in trying to break Nvidia’s stranglehold on the AI sector, where most AI applications are written by default to run on Nvidia’s chips or GPUs.
PyTorch is the popular open source toolkit that AI application developers and startups use to build, train, and run AI models. It is, by some accounts, used by 85% of AI applications. PyTorch was originally developed by Meta (then Facebook) in 2016, long before the world had any idea that LLMs were going to change the world. At the time, Nvidia GPUs were already the standard hardware for AI research, so many of PyTorch’s features were designed to work specifically with them. In 2022, Meta turned PyTorch over to the Linux Foundation, a nonprofit organization that runs many important open source projects.
Google’s version, TorchTPU, allows AI applications to run on Google’s AI chips, called Tensor Processing Units (TPUs). This also makes it easier to migrate existing PyTorch applications to TPUs and, therefore, to Google Cloud. Developers can do this with a simple configuration change and without rewriting their applications.
Amazon made a similar announcement on Monday, releasing its own version of PyTorch, called TorchNeuron. TorchNeuron allows AI applications to be ported to Amazon’s AI chip, Trainium, and to AWS without rewriting. Amazon also said it was going even further in its efforts to loosen Nvidia’s grip on PyTorch. It submits code to the main PyTorch project which decouples various features from Nvidia chips, making it more open to any hardware by default.
Google could outperform Amazon in several ways, however. He says he worked directly with Meta, Hugging Face, Inferact (creator of vLLM) RadixArk (creator of SGLang), and others to ensure that TorchTPU’s features and code libraries work well with these popular AI development tools, just like the original PyTorch does.
Additionally, the search giant launched a dedicated tool, the TPU Agent Suite, to help developers migrate their applications to Google’s chips and cloud and fix errors along the way. It even updated its popular LiteRT tool, which runs AI on phones, laptops and embedded devices, to work better with PyTorch AI applications.
None of this means that AI application developers will automatically abandon Nvidia’s hardware en masse just because they can now easily migrate to alternative chips and clouds. But Amazon and Google are clearly working to end Nvidia’s advantage.
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.
Gn bussni
