Apache Airflow

workflow management, data orchestration, etl
Open jobs

1850

Companies looking for Airflow

1710

Apache Airflow: the workflow automation darling of the data world, born at Airbnb in 2014. It’s open-source, meaning free, but don’t let that fool you—it’s no walk in the park. Competing with the likes of Luigi, Prefect, and cloud-native solutions, Airflow's main draw is its DAG-based approach (Directed Acyclic Graphs), which gives you full control over complex pipelines. In other words, it’s a power tool for data engineers who love Python and have a thing for manually tweaking dependencies.
In its current state, Airflow dominates orchestration, especially in companies with mature data engineering teams. But beware: Airflow is highly flexible, which also means highly finicky. Miss one task or get the scheduling wrong, and your whole pipeline could come crashing down at 3 a.m. The UI is functional but looks like it escaped from 2015, and scalability? Well, you'll be tuning it yourself. It’s powerful but requires constant attention—like a temperamental race car.

Used together with Airflow

Additional Resources

Compare to other orchestration tools
Jobs (this month)

1850

Companies with Jobs

1710

Jobs in using Apache Airflow for but please no

data engineer

Data Engineer II - QuantumBlack, AI by McKinsey (Critical Industries) @ quantumblack

US | 2025-12-27
Data Engineer II role promises to design, build, and optimize modern data platforms powering analytics and AI, with a tour through streaming and batch pipelines across aerospace, utilities, and... read more »
Analytics, AI/ML, R, Data Streaming, Vector DB, RAG, Agile/Scrum, C, Computer Science, Data Engineering, Python, Scala, Java, SQL, Cloud Computing, AWS, GCP, Azure, Oracle, Snowflake, BigQuery, Redshift, Delta, Databricks, AWS Glue, dbt, Spark, Flink, Kafka, Kinesis, Airflow, Dagster, Prefect, CI/CD, Terraform, CloudFormation, DataOps, Datadog, Prometheus, Amazon SageMaker, MLOps, GenAI, LLM, Management
data engineer

Data Engineer @ fanduel

US | 2025-12-26 | USD 116000 - 152250 / year
FanDuel's data engineer role reads like a standard pipeline gig: build and maintain batch and streaming jobs, code in Python and SQL, and keep Databricks, Spark, Airflow, and dbt humming. It... read more »
Data Engineering, Analytics, AI/ML, Data Streaming, Python, SQL, Spark, Agile/Scrum, Data Quality, Analytics Engineering, Java, Scala, Databricks, Airflow, dbt, Kafka, Data Modelling, DWH, ETL/ELT, Cloud Computing, AWS, GCP, Azure, BI, Data Science, Git, CI/CD, Data Governance
promoted

O'Reilly: Building AI Agents with Model Context Protocol (MCP)

Design and implement composable agent architectures using MCP. Understand the MCP architecture and how it enables AI applications to access external context. Build MCP servers that expose tools, resources, and prompts to LLMs.

Get Started
data engineer

Senior Engineer - Data Analytics @ geico

US | 2025-12-25 | USD 100000 - 230000 / year
GEICO is seeking a Senior Engineer to develop high-performance data pipelines and models, with a focus on building resilient distributed systems and supporting internal analytics. The role's... read more »
Data Analytics, Analytics Engineering, Data Modelling, Python, SQL, NoSQL, Spark, dbt, Docker, Kubernetes, Azure, Power BI, Superset, Big Data, Kafka, Git, Snowflake, dimensional modeling, Analytics, BI, Iceberg, Airflow, ETL/ELT, CI/CD, DevOps, Azure DevOps, Management, API, Data Quality, Marketing, AWS, GCP, Cloud Computing, Databricks, Computer Science, Data Science
data engineer

Data Engineer -601/602 @ ptrglobal

US | 2025-12-24 | USD 65 - 70 / hour
A data engineer role focused on AWS, Python, and Spark within a bank's consumer division; requires designing data pipelines and models, with a differentiator being expertise in cloud data lake... read more »
AWS, Python, Spark, Data Lake, Agile/Scrum, Data Collection, Analytics, SQL, NoSQL, Data Quality, Data Governance, Data Engineering, PySpark, GenAI, API, Cloud Computing, Data Lakehouse, Databricks, Hadoop, PostgreSQL, Oracle, Cassandra, DynamoDB, MongoDB, Snowflake, Redshift, Airflow, Unix, Avro, Protobuf, Parquet, Iceberg, Data Streaming, Data Modelling, Data Vault, dimensional modeling, CI/CD
data engineer

Data Engineer II - QuantumBlack, AI by McKinsey @ quantumblack

US | 2025-12-22
Data Engineer II role promises high impact across clients and a culture of continuous learning, but the job description reads like a curriculum vitae for a consulting fortress: heavy emphasis on... read more »
Analytics, AI/ML, R, Data Streaming, Vector DB, RAG, Agile/Scrum, C, Computer Science, Data Engineering, Python, Scala, Java, SQL, PySpark, Cloud Computing, AWS, GCP, Azure, Oracle, Snowflake, BigQuery, Redshift, Delta, Databricks, AWS Glue, dbt, Spark, Flink, Kafka, Kinesis, Airflow, Dagster, Prefect, CI/CD, Terraform, CloudFormation, DataOps, Datadog, Prometheus, Amazon SageMaker, MLOps, GenAI, LLM, Management