Google Cloud Run Task Runner
Available on: >= 0.17.0
Run tasks as containers on Google Cloud Run.
The containerImage property is required because Cloud Run executes tasks as containers. You can use any image from a public or private registry.
Your service account needs to have the following IAM roles attached to use the service:
- Cloud Run Developer
- Logs Viewer
- Storage Admin (to upload files to GCS and download files from GCS)
- Owner/Editor of the Compute Engine default service account (to be able to provision compute resources for the Cloud Run container)
To use inputFiles, outputFiles or namespaceFiles properties, make sure to set the bucket property. The bucket serves as an intermediary storage layer for the task runner. Input and namespace files will be uploaded to the cloud storage bucket before the task run. Similarly, the task runner will store outputFiles in this bucket during the task run. In the end, the task runner will make those files available for download and preview from the UI by sending them to internal storage. To make it easier to track where all files are stored, the task runner will generate a folder for each task run. You can access that folder using the {{bucketPath}} Pebble expression or the BUCKET_PATH environment variable.
Due to the ephemeral nature of Cloud Run, the task runner will not run the task in the working directory but in the root directory. Therefore, you have to use the {{workingDir}} Pebble expression or the WORKING_DIR environment variable to access the inputFiles and namespaceFiles from the task's working directory.
Note about termination of compute resources:
- when the Kestra Worker running this task is terminated, the Cloud Run Job will still run until completion — this is intended to avoid your tasks being interrupted due to Worker crashes.
- when you manually kill the execution from the UI, the Cloud Run Job will be terminated — this is intended to avoid unnecessary costs when you explicitly stop the execution (Note that this behavior is a work-in-progress, you can track the state here on GitHub).
Examples
The following example runs a simple Shell command in a Cloud Run container:
id: new-shell
namespace: company.team
tasks:
- id: shell
type: io.kestra.plugin.scripts.shell.Commands
taskRunner:
type: io.kestra.plugin.gcp.runner.CloudRun
projectId: "{{ vars.projectId }}"
region: "{{ vars.region }}"
commands:
- echo "Hello World"
The following example runs a Shell command in a Cloud Run container and passes input files to the task:
id: new-shell-with-file
namespace: company.team
inputs:
- id: file
type: FILE
tasks:
- id: shell
type: io.kestra.plugin.scripts.shell.Commands
inputFiles:
data.txt: "{{inputs.file}}"
outputFiles:
- out.txt
containerImage: centos
taskRunner:
type: io.kestra.plugin.gcp.runner.CloudRun
projectId: "{{ vars.projectId }}"
region: "{{ vars.region }}"
bucket: "{{ vars.bucker }}"
commands:
- cp {{ workingDir }}/data.txt {{ workingDir }}/out.txt
For a full list of properties available in the CloudRun task runner, check the GCP plugin documentation or explore the same in the built-in Code Editor in the Kestra UI.
Was this page helpful?