What Are TPUs? A Guide to Tensor Processing UnitsWhat Are TPUs? A Guide to Tensor Processing UnitsWhat Are TPUs? A Guide to Tensor Processing Units

Forget GPUs. If you’re serious about AI hardware, you’ll train and serve models using TPUs. At least, that’s what you would do if you were Google – and, possibly, if you’re a Google Cloud customer. For everyone else, TPUs aren’t actually a viable AI hardware solution because Google doesn’t sell the physical devices to third parties.

That’s the short take on the role that TPUs play in AI workloads . For the details, keep reading as we unpack everything you need to know about TPUs in the data center.

What Is a TPU?

A TPU – short for tensor processing unit – is a type of computing chip optimized for training and serving certain types of AI models. More specifically, TPUs are a form of application-specific integrated circuit, or ASIC. An ASIC is any type of chip designed for a specific task. In the case of TPUs, that task is AI workloads. TPUs fall under the ‘AI accelerator’ category of specialized hardware designed to optimize machine learning workloads.

Google began developing TPUs in 2015 for use in its own AI projects. Starting in 2018, it made them available to other businesses, primarily by offering TPU-powered cloud server instances on Google Cloud. We’ll explain how to access TPUs below.

How Do TPUs work?

Because TPUs are a proprietary product developed by Google, full details on how they work are not publicly available. At a high level, however, they use a design that places an AI model’s data and parameters into a matrix, then processes them in parallel.

This approach is beneficial for AI workloads that use deep learning or reinforcement learning – the methods that power most of the major large language models (LLMs) available today. Thus, although Google began working on TPUs several years before the onset of the current generative A I and agentic AI craze, TPUs turned out to be very good at supporting the models at the heart of modern AI technology.

What Are TPUs used for?

TPUs can perform two key tasks for AI data centers:

TPUs vs. GPUs

TPUs are similar to graphics processing units , or GPUs, which can also be used to train and serve AI models. Both types of chips are specialized hardware that excels at supporting AI.

However, there are several key differences between TPUs and GPUs:

Which Versions of TPUs are Available?

Since 2015, Google has released seven major versions of its TPU products. According to Google, each new release has improved speed and energy efficiency.

In addition, Google has typically promoted new TPU products as solutions that keep pace with evolving AI needs. For instance, it says that its latest TPU, TPU v7 (Ironwood), is optimized for proactive information generation .

Who Needs TPUs?

In general, any team developing AI models can potentially benefit from TPUs to accelerate model training and inference. TPUs may be able to complete AI workflows faster than GPUs. Google also touts TPUs as being energy efficient , which is not always the case with other types of AI hardware.

That said, the fact that third parties can’t install and monitor TPUs directly makes it challenging to know with certainty how energy-efficient they are in different scenarios. It’s also unclear to what extent TPUs themselves are inherently energy-efficient, versus the extent to which they operate in data centers that use innovative cooling techniques, like liquid cooling , to save energy. Google has stated that it uses liquid cooling for TPUs.

TPUs are particularly well-suited for workloads involving LLMs, natural language processing, computer vision tasks, and recommendation systems. Organizations running complex transformer-based models like BERT or T5, or those requiring high-throughput inference for services like real-time translation or content moderation, may find TPUs especially beneficial.

Google's own services like Search, Photos, and Maps leverage TPUs for their AI capabilities.

Read more of the latest data center hardware news

Where Can I Buy a TPU?

If you want to purchase a TPU, the bad news is that you probably can’t acquire the type of TPU you’re looking for. Most of the TPUs developed by Google are available only as IaaS products through Google’s Cloud TPU service – which is the equivalent of a GPU-as-a-Service offering, except it provides access to TPUs instead of GPUs.

This means you can rent servers equipped with TPUs and use them for AI training and inference. But you can’t install TPUs in your own server or data center.

There is one exception: A product known as Edge TPU, a less powerful version of the TPUs running in Google Cloud data centers. You can purchase Edge TPUs through Coral , an AI company owned by Google. Edge TPUs can be useful for training or running models on local computers or specialized edge hardware, but they’re not a substitute for data center-grade AI chips.

• ITPro Today's 2025 IT Priorities Report

  Aug 8, 2025
  |

  2 Min Read

• Edge Computing Trends: Adoption, Challenges, and Future Outlook

  Jul 15, 2025
  |

  2 Min Read

• ITPro Today’s 2024 State of DevOps Report

  Dec 16, 2024
  |

  2 Min Read

• BCDR Basics: A Quick Reference Guide for Business Continuity & Disaster Recovery

  Oct 10, 2024
  |

  1 Min Read