Orchestrate Python Code: Data Engineering Zoomcamp - 2.2.3
Now that we've built our first workflow, we can take it a step further by adding Python code into our flow. In Kestra, we can run Python code from a dedicated file or write it directly inside of our workflow. π Try Kestra for free: https://go.kestra.io/yt/try-kestra π¨βπ» Check out the Data Engineering Zoomcamp GitHub Repository here: https://go.kestra.io/de-zoomcamp Kestra is an open-source, event-driven orchestration platform that simplifies building both scheduled and event-driven workflows. By adopting Infrastructure as Code practices for data and process orchestration, Kestra enables you to build reliable workflows with just a few lines of YAML. Timestamps 0:00 - Introduction 0:23 - Examples can be found on GitHub 0:36 - Python Task Types 1:35 - Docker Task Runners 2:26 - Process Task Runners 3:01 - Kestra Python Package 3:44 - Summary ---------- π Read the documentation: https://go.kestra.io/docs β Start your journey with Kestra: https://go.kestra.io/github π¬ Join the Kestra Community: https://go.kestra.io/slack