At Google Cloud Next on April 22, Google split its eighth-generation TPU into two chips: the TPU 8t for model training and the TPU 8i for inference. Google says the chips offer up to 3x faster training and 80% better performance per dollar.
Google still sells Nvidia systems alongside its own silicon and promises its cloud will have Nvidia's latest Vera Rubin chip later this year. Specialization reflects how different training and inference workloads have become, as inference grows into the larger share of AI compute demand. The performance figures are Google's own claims.
Source: CNBC