Apache Airflow

workflow management, data orchestration, etl
Open jobs

1850

Companies looking for Airflow

1710

Apache Airflow: the workflow automation darling of the data world, born at Airbnb in 2014. It’s open-source, meaning free, but don’t let that fool you—it’s no walk in the park. Competing with the likes of Luigi, Prefect, and cloud-native solutions, Airflow's main draw is its DAG-based approach (Directed Acyclic Graphs), which gives you full control over complex pipelines. In other words, it’s a power tool for data engineers who love Python and have a thing for manually tweaking dependencies.
In its current state, Airflow dominates orchestration, especially in companies with mature data engineering teams. But beware: Airflow is highly flexible, which also means highly finicky. Miss one task or get the scheduling wrong, and your whole pipeline could come crashing down at 3 a.m. The UI is functional but looks like it escaped from 2015, and scalability? Well, you'll be tuning it yourself. It’s powerful but requires constant attention—like a temperamental race car.

Used together with Airflow

Additional Resources

Compare to other orchestration tools
Jobs (this month)

1850

Companies with Jobs

1710

Jobs in using Apache Airflow for but please no

machine learning engineer

Senior Machine Learning Engineer - (Vendor Data) @ delivery-hero-se

DE | 2025-12-28
Delivery Hero’s Senior Machine Learning Engineer for Vendor Data promises end‑to‑end ML at scale across 70+ markets, designing and operating production systems that balance performance with cost.... read more »
AI/ML, Data Science, Cloud Computing, Data Management, MLOps, DataOps, TensorFlow, PyTorch, AWS, GCP, Azure, Docker, Kubernetes, CI/CD, Airflow, Grafana
machine learning engineer

Machine Learning Engineer @ scribd

US | 2025-12-26 | USD 126000 - 196000 / year
Scribd's Machine Learning Engineer II sits on the Orion platform, building and tuning real-time ML systems that power recommendations and AskAI. Expect end-to-end ownership—from ingestion and... read more »
Spark, AI/ML, GenAI, Python, Go, Scala, Airflow, Databricks, AWS, Amazon SageMaker, LLM, API, Cloud Computing, AWS Lambda, CloudWatch, Datadog, Terraform, Azure, GCP, Computer Science
promoted

BigQuery Cost Optimization Guide by Masthead Data

The BigQuery Cost Optimization Guide is a valuable resource that offers a deep dive into understanding and managing the complexities of BigQuery costs. It provides practical guidelines and strategies to help you optimize both storage and compute expenses, ensuring you get the most value out of...

Learn More
machine learning engineer

Machine Learning Engineer @ scribd

US | 2025-12-26 | USD 103500 - 196000 / year
Machine Learning Engineer II at Scribd, built to design, deploy, and optimize near real-time ML for millions across the Orion platform. Expect heavy exposure to an embedding-based retrieval stack... read more »
Spark, AI/ML, GenAI, Python, Go, Scala, Airflow, Databricks, AWS, Amazon SageMaker, LLM, API, Cloud Computing, AWS Lambda, CloudWatch, Datadog, Terraform, Azure, GCP, Computer Science