Cloud Run Jobs vs. Cloud Batch: Choosing Your Engine for Run-to-Completion Workloads

Google Cloud offers plenty of different products and services, some of which seem to be covering overlapping needs. There are multiple storage solutions ( Cloud Storage , Filestore ), database products ( Cloud SQL , Spanner , BigQuery ) or ways to run containerized applications ( Cloud Run and GKE ). The breadth of options to choose from can be overwhelming and lead to situations where it’s not obvious which way to go to achieve your goal. Similar situation applies to offline processing (aka batch processing). This is a situation where you have some data and want to run the same operation on each piece of this data. For example: transcoding a big video collection, resizing an image gallery or running inference against a prepared set of prompts. The recommended way to handle such situations is to use proper tools that will automatically scale, handle errors and guarantee that all data has been processed. Cloud Batch and Cloud Run Jobs are two of the options to consider when you want to