Prefect

workflow management, data orchestration, etl
Open jobs

179

Companies looking for

174

Let’s be honest, the data engineering world was already overflowing with workflow schedulers when Prefect sauntered in, promising to be more than just a scheduler. It arrived with a vision of data pipelines as code, embracing Python’s dynamic nature and a frankly admirable commitment to observability. The pitch? Move beyond brittle cron jobs and embrace a truly modern, developer-first approach.

In practice, it’s a bit more involved than simply “wiring things up in Python.” While the core concepts are elegantly simple, mastering Prefect requires a shift in thinking – embracing its task and flow definitions. It distinguishes itself from Airflow by leaning heavily into dynamic mapping and a more opinionated, Python-native experience. Today, it’s gaining traction among those tired of Airflow’s complexity and those building genuinely dynamic data workflows, though one suspects a significant portion of its user base are Python developers who just wanted an excuse to write more Python.

Used together with Prefect

Additional Resources

Compare to other orchestration tools
Jobs (this month)

179

Companies with Jobs

174

Jobs in using Prefect for but please no

data engineer

Senior Data Engineer @ sensorfact

NL | 2025-12-25
Sensorfact’s Senior Data Engineer role reads like a battle plan for industrial IoT data: build real-time streams from raw sensor data, detect faults on the fly, and serve personalised guidance via... read more »
AWS Glue, Analytics, AI/ML, MLOps, Cloud Computing, AWS, Kafka, Flink, ClickHouse, Prefect, AWS Lambda, API, Agile/Scrum, IoT, PhD, Computer Science, Python, Data Streaming, Spark, Kubernetes, Docker, Scala, Airflow, Git, CI/CD, PostgreSQL, Redshift, BigQuery
data engineer

Sr. Product Manager, Lakeflow @ databricks

NL | 2025-12-25
Databricks seeks a Product Manager for Lakeflow Jobs to own data-aware orchestration on a platform that supposedly handles hundreds of millions of workloads weekly with near-perfect reliability.... read more »
Databricks, AI/ML, ETL/ELT, BI, Data Streaming, Git, Computer Science, Management, Airflow, Dagster, Prefect, Spark, dbt, SQL, Python, Analytics, Data Lakehouse, Delta
promoted

O'Reilly: Building AI Agents with Model Context Protocol (MCP)

Design and implement composable agent architectures using MCP. Understand the MCP architecture and how it enables AI applications to access external context. Build MCP servers that expose tools, resources, and prompts to LLMs.

Get Started
data engineer

Senior Data Engineer, Data Platform @ manychat

NL | 2025-12-22
The role is a compelling opportunity for a data engineer passionate about building and evolving a robust data platform. The position emphasizes platform development, innovation, and scalability,... read more »
Marketing, Analytics, AI/ML, Data Quality, Data Modelling, Data Engineering, Cloud Computing, AWS, Snowflake, Redis, dbt, Dagster, Prefect, Monte Carlo, SQL, ETL/ELT, Python, DWH, DevOps, Git, CI/CD, Docker, Kubernetes