Introducing multi-cluster GKE Inference Gateway: Scale AI workloads around the world

The world of artificial intelligence is moving fast, and so is the need to serve models reliably and at scale. Today, we're thrilled to announce the preview of multi-cluster GKE Inference Gateway to enhance the scalability, resilience, and efficiency of your AI/ML inference workloads across multiple Google Kubernetes Engine (GKE) clusters — even those spanning different Google Cloud regions. Built as an extension of the GKE Gateway API , the multi-cluster Inference Gateway leverages the power of multi-cluster Gateways to provide intelligent, model-aware load balancing for your most demanding AI applications. Why multi-cluster for AI inference? As AI models grow in complexity and users become more global, single-cluster deployments can face limitations: Availability risks: Regional outages or cluster maintenance can impact service. Scalability caps: Hitting hardware limits (GPUs/TPUs) within a single cluster or region. Resource silos: Underutilized accelerator capacity in one cluster ca