​Task ​Runners

Task Runners are plugins capable of executing your tasks in remote environments.

Many data processing tasks are computationally intensive and require a lot of resources (such as CPU, GPU, and memory). Instead of provisioning always-on servers, Task Runners can run your code on dynamically provisioned compute instances in the cloud, such as AWS ECS Fargate, Azure Batch, Google Batch, auto-scaled Kubernetes clusters, and more.

Task Runner Overview

Task Runner capabilities and supported plugins.

Task Runner Benefits

How Task Runners can help with resource allocation and environment management.

Task Runner Use Cases

Use cases that can benefit from Task Runners.

Task Runner vs. Worker Group

Find out when to use task runners or worker groups.

Task Runner Types

This section lists all task runners available in Kestra.