Data workflows/automation often start as something small. Like asking an Engineer to write a few scripts. It can soon blow up in the face of many companies and the constant needs and scale change.

To tackle all that a data orchestration tool is important. The open source community-backed Apache Airflow has helped me not just scale my data pipelines but also my career.

In this talk, I want to introduce you to the tool I love the most and take you on my open-source journey with Apache airflow, which helped me embark on different avenues in devops and data engineering practices