Prefect just bought Dagster. Two of the biggest names in orchestration are now one company, and the reasons go well beyond a slow grower getting folded into a faster one.
If you're trying to work out whether your orchestration tool is about to get sidelined, or whether "agent governance" is really just a new label on an old data stack problem, this one's for you.
David Yaffe is co-founder and CEO of Estuary, a real-time data integration platform, and previously ran product at Invite Media (acquired by Google) and co-founded Arbor (acquired by LiveRamp). He's spent his career building the pipelines and infrastructure that data teams actually run on.
Ross and David break down what the Prefect-Dagster deal really signals: a shift in buyer from data engineers to IT and the CIO, and a race to own agent governance before anyone else does. You'll walk away with a sharper read on where orchestration is heading, why documentation might beat MCP servers for agent-ready systems, and where the next round of data stack consolidation is likely to land.
Expect a candid take on M&A positioning versus reality, the mechanics of agent permissions and observability, and a debate on MCP servers versus CLI documentation for building agent-ready infrastructure. This is for data and analytics leaders trying to separate genuine strategy from press-release spin, not for anyone after a puff piece on the deal.
Key Takeaways
- The Prefect-Dagster deal has less to do with orchestration and more to do with who owns agent governance and permissions once agentic workflows go mainstream.
- The buyer for agent governance tooling isn't the data engineer scheduling jobs. It's the IT department, and getting those two buyers speaking the same language won't be simple.
- Well-documented CLIs can outperform MCP servers for agent use, cutting token spend and doing double duty for human users too.
- Treat every agent like a new graduate on day one. If your documentation, schemas, and data model aren't clear enough for a novice to follow, an agent will struggle just as much, only more expensively.
Chapter Markers
00:00 Prefect's acquisition of Dagster explained
01:39 Welcome David Yaffe, Estuary co-founder
02:02 First reactions to the Prefect-Dagster deal
03:46 Why orchestration alone isn't a sticky business
07:19 Fast MCP and the rise of agent governance
09:47 MCP servers versus CLI documentation debate
12:52 Treating agents like new hires on day one
14:41 Who actually buys agent governance tools
17:40 The "aqua-pivot": why Prefect really bought Dagster
20:07 What happens to existing Dagster users
24:17 Where data industry consolidation happens next
27:27 Governance, liability and the human parallel
29:50 Time-boxed access and the future of agent permissions
31:20 What success looks like a year from now
35:07 What We're Watching: John von Neumann and the death of reading
40:14 Wrap up and final thoughts
Useful Links & Resources
- David Yaffe on LinkedIn: https://www.linkedin.com/in/davidyaffe
- Estuary (David's company): estuary.dev
- The Maniac by Benjamin Labatut, referenced in the John von Neumann discussion
Connect With the Show
- Ross Katz on LinkedIn: https://www.linkedin.com/in/b-ross-katz/
- Ross Katz on X: https://x.com/brosskatz
- CorrDyn on LinkedIn: https://www.linkedin.com/company/corrdyn/
- corrdyn.com
Got a take on the Prefect-Dagster deal, or a horror story about agent permissions gone wrong? Tell us in the comments. We're curious whether your team sees this as consolidation, or the opening move in the fight for agent governance.
If you want to talk about your data challenges, or you think we got something wrong, find us at corrdyn.com.
#EventualConsistency #DataInfrastructure #AIagents #DataStack #Orchestration
