Hugging Face adds native TPU support to Transformers and Diffusers
Transformers and Diffusers now support TPU through TorchTPU, which adds another backend to PyTorch. According to Hugging Face, Optimum TPU is no longer needed to run them on TPU. However, some tests are not yet passing.
When prototyping, you can load a model directly onto TPU or move an image pipeline to it using the parameters and calls shown.