Deploying Prefect Workflows on Cloud Run with Cloud SQL (Production-Ready GCP Setup)

If you're running data workflows and don’t want to manage Kubernetes, this architecture might help. In this project, I built a production-ready workflow system using: Prefect for orchestration 🔄 Cloud Run for serverless container execution 🚀 Cloud SQL (PostgreSQL) for metadata storage 🗄️ IAM-based authentication between services 🔐 🏗️ Architecture Overview Prefect Agent → Cloud Run Container → Cloud SQL (Private IP) Why this approach works well: Fully serverless compute ⚡ Auto scaling No VM maintenance Secure database connectivity Clean separation of orchestration and execution 🛠️ Key Implementation Steps Create Cloud SQL instance (Private IP) Build Docker image for Prefect flow Push image to Artifact Registry Deploy to Cloud Run Configure IAM roles properly Connect Cloud Run to Cloud SQL ⚠️ Common Mistakes Forgetting Cloud SQL Auth Proxy Not configuring VPC connector Using public IP for database 🎯 Ideal Use Cases ETL pipelines Background microservices Event-driven automation Internal w