P
Paolo Perrone
Guest
Durable execution saves the result of every step your agent finishes, so a crash costs you the one step that was running and not the hour of API calls behind it. LangGraph checkpoints graph nodes. DBOS, a library that keeps the workflow log in your own database, checkpoints into the Postgres you already run. Inngest replays saved step results. Temporal replays your workflow code against an event history. What separates them is where that state lands: your own database, a cluster you run, or somebody else's platform.
You are inside the
Agents course Every lesson free, in order.
All four resume a crashed agent. What you get back is whatever you remembered to wrap.
New here? Every Friday, one production decision taken apart like this. Subscribe free.
Your research agent runs forty steps. If that loop is new to you, we took it apart in What is an AI Agent?. It searches, opens eleven pages, summarizes each one, cross-checks fifteen claims against the sources, and writes a report. Step 38 is the last cross-check. Somewhere in there you have burned about thirty dollars in tokens and eleven minutes of wall clock.
At step 38, the pod gets evicted.
The agent restarts. It searches again, and the search does not return the same eleven pages, because two of the results have moved since the first search. The agent summarizes the new set. Eleven minutes later it reaches step 38, the box is still short on memory, and it gets killed in the same place.
The thirty dollars is the part you can see. What you cannot see is that the report changed. Neither run wrote down which eleven pages it used, so nothing tells you which pages went into the report your user read, or which ones the dead run would have used.
Agents run for minutes, spend most of that waiting on somebody else’s API, and hold everything they have learned in memory the whole time. Your HTTP handlers have none of the three, and your cron jobs have the first two. A retry that starts from zero costs you a second full run at full price, and the third one costs the same again.
When a background job fails, it starts again from the beginning. A durable execution tool writes down each step as it finishes, so the restart costs you the one step that died and the eleven summaries stay bought.
LangGraph, DBOS, Inngest and Temporal all do that, and where each one writes that record is what constrains you. LangGraph writes to a checkpointer you configure. DBOS writes to the Postgres your product already uses. Inngest writes to Inngest’s own platform. Temporal writes to a cluster somebody runs.
In one sentence: if your agent is a LangGraph graph, crash recovery is a checkpointer object you pass to
If you built the agent in LangGraph, you have a checkpointer, and it saves the graph’s state after every node the graph runs. Point that checkpointer at Postgres, pass it to
The good:
The bad:
Best for: an agent that is already a graph, where the failure you are defending against is process death rather than a flaky vendor API.
The ceiling: the first time a graph needs to wait three days for a human to approve something. A checkpointer holds state, it does not hold a schedule.
Do not pick this: if you are multi-tenant and every customer shares one rate limit, because you will build a scheduler on top of the graph and end up maintaining both.
In one sentence: a hosted platform where you wrap work in
Wrap work in
The good:
The bad:
Best for: a team that wants retries, per-tenant concurrency and traces on day one and has nobody to spare for running a queue and the workers behind it. Wrap your first step
The ceiling: the function that gets expensive to enter. Every step you add is one more full pass through everything sitting outside a step boundary.
Do not pick this if you are in a regulated vertical where a compliance review of a new subprocessor takes longer than building the queue yourself.
In one sentence: durable execution as a library, checkpointing into the Postgres you already have.
You annotate your functions, and DBOS writes each workflow input and step output into its own tables inside that Postgres, queues included. There is no orchestration server: the library polls those tables itself, and every process running DBOS can pick up work.
The good:
The bad:
Best for: a team with an existing Postgres, a strong preference for owning its own state, and workloads where transactional consistency between business data and workflow progress is the point.
The ceiling: the day you move off DBOS, because the workflow history is in your database and migrating it is your job.
Do not pick this if your Postgres is already the thing you worry about on a busy day, and understand going in that the workflow history becomes yours to migrate if you ever leave.
In one sentence: the industrial option, and the only one here that expects you to run a cluster.
Temporal records every workflow decision in an event history. On recovery it re-executes your workflow code from the top and compares the commands your code issues against that history. Match, and execution moves forward. Mismatch, and you get a non-determinism error
The good:
The bad:
Best for: long-running, high-value workflows where a lost run is a business incident and the team already has platform engineers.
The ceiling: it does not have one. That is the point: the cluster, the worker fleet and the determinism rules are what hold the ceiling open, and you install all three before the first workflow runs.
Do not pick this with three engineers and nobody on platform. The cluster does not care that you were busy, and the determinism rules will reject a workflow written by somebody who has not read them.
I ran all four. Same agent, same eleven pages, real Postgres, a real Temporal server, a real Inngest dev server, and a
The
All four hold. That is the finding, and it is less useful than it sounds, because none of them holds by default.
LangGraph needed
Every one of those is a line of setup, and every one of them is invisible until you kill something.
There is a model call in your agent that a crash bills you for twice, and you do not know which one it is.
Four questions, in this order.
The tree below shows where each answer sends you.
Most readers stop at the first question. If your agent is already a graph and no shared rate limit has to be rationed between tenants, attach a checkpointer, set durability to
Question three is the only one you do not get to decide. Compliance either lets step payloads leave your network or it does not, and a yes there removes Inngest before you have compared a single feature. If payloads can leave your network, you can have Inngest running this afternoon, with no queue and no cluster.
The table below is the same four decisions as a reference you can screenshot.
Note: read the third column first. “Best for” is where every comparison spends its words and it is the column that changes least, because all four are good at the thing they exist to do. What differs is who each one locks out, and that column is the one your compliance team, whoever owns the database, and your headcount decide for you. The setup times measure one engineer getting a first durable workflow running in staging. A migration takes longer.
They pick a durable layer and keep the same step boundaries. Wrapping your whole agent in one
They never test recovery. Nobody has killed the workflow mid-run, so the step boundaries in your repo are still a guess about what the tool saves. I killed all four while writing this, and the tables above are what came back. Your own step boundaries are still a guess until you do the same, which is exactly why the probe ships with them. You have the same gap in the retry logic and the eval suite, which is what Why AI Agents Keep Failing in Production is about. The failure is always the same shape: something expensive was outside a boundary, and nobody found out until the bill.
They pick for month one and pay in month nine. A workflow that started in March has to finish under the code you ship in August, and by then you have renamed two steps and reordered a branch. Temporal makes you declare that change and version the workflow around it. Inngest keys memoization on the step name, so a rename orphans every in-flight run. LangGraph and DBOS leave the question to you. Nobody asks this in the evaluation, and everybody meets it eventually. A resumed run that answers differently is an eval failure before it is a durability failure, and What is an Eval? covers how you catch it.
They confuse durability with idempotency. Durable execution promises never to re-run a step that reported success. It promises nothing about a step that charged a card and then timed out before reporting anything. You still solve that one.
All four resume a crashed function, and none of them will be the reason your agent works. Watch what each one adds around the resumption instead. That is what you will actually be living with in year two.
Rule out a graph, rule out your own Postgres, rule out a cluster, and Inngest is what is left. Inngest also paid to be here. Both of those are true at once.
If you are starting an agent today and you want one answer rather than four: build it as a graph, attach a Postgres checkpointer, set durability to
The tool is not what decides whether a crash costs you thirty dollars or seventy-five cents. Your step boundaries decide that, and you draw those yourself in either case. Go find out what it costs you
What killed your last long-running agent, and did it resume or restart? Tell me in the comments.
Leave a comment
Is durable execution the same as a job queue?
No. It saves progress inside one unit of work. Your existing queue retries the whole unit, so the agent that died on step 38 buys all 38 of them a second time. You can build resumption on top of a queue, and that build is the project all four of these tools replace.
Do I need this if my agent finishes in ten seconds?
Probably not. The value scales with how much work sits between the start and the crash. A ten-second agent that fails costs you ten seconds. Wrap it in a retry and move on. The threshold is roughly where a full restart starts costing real money or real minutes.
Can I add durable execution to an agent I already wrote?
Yes, and the work is drawing step boundaries rather than rewriting logic. You wrap each expensive call in whatever the tool’s step primitive is. Temporal is the exception, because determinism is a constraint on the workflow function itself, so an agent that reads the clock or calls an API inline needs that code moved into Activities first.
1 - Durable execution, LangChain docs
2 - How Inngest functions are executed, Inngest docs
3 - DBOS Architecture, DBOS docs
4 - Workflow definition, Temporal docs
You are inside the
Agents course Every lesson free, in order.TL;DR
All four resume a crashed agent. What you get back is whatever you remembered to wrap.
- A crash costs you whatever you did not wrap. All four save each step’s result the moment it finishes, so a retry hands back the saved answer instead of running the step again. None of them saves the model call you made outside a step boundary, and that is where the money goes.
- LangGraph writes its checkpoints in the background by default. The
asyncmode persists while the next node runs, so a hard crash can lose the newest one. One config value moves you tosync - DBOS puts the workflow log in your database. Each step’s output commits in the same Postgres transaction as the rows it wrote, so recovery reads that output back and skips the step instead of running it again.
- Inngest runs your function once per step. It re-invokes the whole function body each time and injects saved results into completed
step.run()calls, matched by a hash of the step name. - Temporal replays your workflow code from an event history and compares each command against what the history recorded. A clock, a random number or a direct HTTP call can change that command on the second pass, so none of them are allowed in workflow code.
- Paid subscribers get
boundaries.pyandprobe.py. One reads your agent and names every model call a crash makes you buy twice, with nothing installed. The other kills a real run and counts what re-executed. Both ship with four reference implementations.
New here? Every Friday, one production decision taken apart like this. Subscribe free.Your Agent Died at Step 38
Your research agent runs forty steps. If that loop is new to you, we took it apart in What is an AI Agent?. It searches, opens eleven pages, summarizes each one, cross-checks fifteen claims against the sources, and writes a report. Step 38 is the last cross-check. Somewhere in there you have burned about thirty dollars in tokens and eleven minutes of wall clock.
At step 38, the pod gets evicted.
The agent restarts. It searches again, and the search does not return the same eleven pages, because two of the results have moved since the first search. The agent summarizes the new set. Eleven minutes later it reaches step 38, the box is still short on memory, and it gets killed in the same place.
The thirty dollars is the part you can see. What you cannot see is that the report changed. Neither run wrote down which eleven pages it used, so nothing tells you which pages went into the report your user read, or which ones the dead run would have used.
Agents run for minutes, spend most of that waiting on somebody else’s API, and hold everything they have learned in memory the whole time. Your HTTP handlers have none of the three, and your cron jobs have the first two. A retry that starts from zero costs you a second full run at full price, and the third one costs the same again.
Where the Saved State Lives
When a background job fails, it starts again from the beginning. A durable execution tool writes down each step as it finishes, so the restart costs you the one step that died and the eleven summaries stay bought.
LangGraph, DBOS, Inngest and Temporal all do that, and where each one writes that record is what constrains you. LangGraph writes to a checkpointer you configure. DBOS writes to the Postgres your product already uses. Inngest writes to Inngest’s own platform. Temporal writes to a cluster somebody runs.
LangGraph
In one sentence: if your agent is a LangGraph graph, crash recovery is a checkpointer object you pass to
compile()If you built the agent in LangGraph, you have a checkpointer, and it saves the graph’s state after every node the graph runs. Point that checkpointer at Postgres, pass it to
compile(), and the graph writes its state to your database from the next run onward. The good:- Zero new infrastructure. The checkpointer writes to a database you already run, and your team calls the same LangGraph API they use for everything else.
- The same saved state that survives a crash also drives human-in-the-loop. You interrupt a graph, show a human the pending action, and resume on approval.
- Three durability modes, so you can trade safety against speed per graph:
syncwrites before the next node starts,asyncwrites while it runs,exitwrites only at the end.
The bad:- The default is
async, which persists in the background while the next node runs. LangChain’s own docs describe the tradeoff as a small risk of data loss if the process crashes. So the checkpoint for step 38 may never reach Postgres. You paid for that cross-check and the restart pays for it again. - LangGraph writes one checkpoint per node, whatever that node does inside itself. Put four tool calls in one node, and the restart runs all four again.
- A checkpointer saves state and does nothing else. It will not retry a failed vendor call, cap one customer at five concurrent runs, or hold you under an API rate limit, so those stay yours to write or you add a queue alongside it
Best for: an agent that is already a graph, where the failure you are defending against is process death rather than a flaky vendor API. The ceiling: the first time a graph needs to wait three days for a human to approve something. A checkpointer holds state, it does not hold a schedule. Do not pick this: if you are multi-tenant and every customer shares one rate limit, because you will build a scheduler on top of the graph and end up maintaining both.Inngest
In one sentence: a hosted platform where you wrap work in
step.run() and Inngest calls your function once per step, skipping the steps that already finished.Wrap work in
step.run(). Inngest then calls your function once per step, handing each call the saved state of the previous one, and the SDK hands back a stored result for any step that already ran instead of executing it a second time. That has one sharp consequence: your function body runs once per step, so anything outside a step runs every single time**.**
The good:- The step boundary is a line in your own repo,
step.run("summarize", …), so you add durability by wrapping a call and redeploying the service you already ship. No worker fleet, no queue to babysit. - Flow control is configuration. A
concurrencykey caps one tenant at five in-flight runs, so one customer spamming the queue cannot bury everyone else’s events behind theirs. Inngest treats that fairness problem as a first-class feature; on the others it is a setting you have to know exists and go find. - The platform already executes your steps, so the traces come from the runtime rather than a second vendor bolted alongside it. We covered what you need to see inside a run in What is LLM Observability?
The bad:- Step inputs and outputs cross a network boundary to Inngest’s platform. Self-hosting exists, so it is not a hard no, but satisfying that constraint means standing up nine services of your own, from the event API to the executor to the dashboard. LangGraph and DBOS meet the same requirement by writing to the Postgres you already run.
- Re-invocation punishes expensive work that sits outside a step. An unwrapped
llm.outline()call runs on every pass, so it costs you once per step in the function, and each pass can produce a different outline while the memoized publish step still holds the first one. The bill is the small half of that. - Step names are the memo key. Rename a step and every run already in flight stops finding its saved result, so those runs execute it again.
Best for: a team that wants retries, per-tenant concurrency and traces on day one and has nobody to spare for running a queue and the workers behind it. Wrap your first step The ceiling: the function that gets expensive to enter. Every step you add is one more full pass through everything sitting outside a step boundary. Do not pick this if you are in a regulated vertical where a compliance review of a new subprocessor takes longer than building the queue yourself.DBOS
In one sentence: durable execution as a library, checkpointing into the Postgres you already have.
You annotate your functions, and DBOS writes each workflow input and step output into its own tables inside that Postgres, queues included. There is no orchestration server: the library polls those tables itself, and every process running DBOS can pick up work.
The good:- Workflow state and your application data sit in the same Postgres, so one
COMMITsaves both. Nothing can crash between them. LangGraph, Inngest and Temporal all write the workflow record outside your transaction, so saving the summary and saving the step are two operations, and you close the gap with an outbox table and a job that drains it - Open source, and the history is a table you can query. Recovery is a
SELECTaway.
The bad:- Workflow traffic lands on the database your product runs on. Queue polling, checkpoint writes and history reads compete with user queries for the same connections.
- The observability you get is the observability you build on top of those tables. There is a console, but the deep trace view of a running agent is not the product’s center of gravity.
- It is the youngest option here. You are picking a layer that will still be holding your workflow history in three years, and DBOS has the shortest track record of the four.
Best for: a team with an existing Postgres, a strong preference for owning its own state, and workloads where transactional consistency between business data and workflow progress is the point. The ceiling: the day you move off DBOS, because the workflow history is in your database and migrating it is your job. Do not pick this if your Postgres is already the thing you worry about on a busy day, and understand going in that the workflow history becomes yours to migrate if you ever leave.Temporal
In one sentence: the industrial option, and the only one here that expects you to run a cluster.
Temporal records every workflow decision in an event history. On recovery it re-executes your workflow code from the top and compares the commands your code issues against that history. Match, and execution moves forward. Mismatch, and you get a non-determinism error
The good:- Activity results are read from history rather than re-executed, so a model call inside an Activity is paid for once no matter how many times the workflow replays. Leave that same call in the workflow function and you buy it again on every replay.
- Versioning is a first-class feature, and Temporal is the only tool here that makes you declare a breaking change rather than discover one.
The bad:- Your workflow code must be deterministic. No
Date.now(), no random, no direct HTTP. Every non-deterministic thing moves into an Activity, which is a real constraint on how an agent gets written and the reason a workflow that ran fine for a month fails on its first replay. - Somebody operates the cluster. Temporal Cloud removes that, and then you are back to a hosted vendor with a different bill.
- Two weeks to production is the honest number, against an afternoon to attach a LangGraph checkpointer.
Best for: long-running, high-value workflows where a lost run is a business incident and the team already has platform engineers. The ceiling: it does not have one. That is the point: the cluster, the worker fleet and the determinism rules are what hold the ceiling open, and you install all three before the first workflow runs. Do not pick this with three engineers and nobody on platform. The cluster does not care that you were busy, and the determinism rules will reject a workflow written by somebody who has not read them.What Each One Actually Repaid
I ran all four. Same agent, same eleven pages, real Postgres, a real Temporal server, a real Inngest dev server, and a
SIGKILL with no cleanup partway through. The model call is stubbed for these runs, so the numbers are about what survives a crash and not about dollars.
The
summarize row carries the answer in each one. Eleven pages, four or five paid for before the crash, and the resume paid for exactly the rest. DBOS shows +1 because the kill landed inside a summary that had started and not yet checkpointed, which is the floor every one of these has.All four hold. That is the finding, and it is less useful than it sounds, because none of them holds by default.
LangGraph needed
durability="sync" and a resume that passes None instead of the original input. Pass the input twice and it merges an empty list over your checkpoint, and every summary you already paid for is thrown away. DBOS needed a stable workflow id, or the restart recovers the old workflow and starts a fresh one beside it: 23 executions where a clean run is 14. Temporal needed the client to attach to the running workflow rather than submit it again, because a killed worker leaves it open and a second submission is rejected. Inngest needed nothing in the code and everything in the timing, since it fans the summaries out in parallel and the whole run finishes in about five seconds.Every one of those is a line of setup, and every one of them is invisible until you kill something.
There is a model call in your agent that a crash bills you for twice, and you do not know which one it is. boundaries.py reads your code and names it in about a second, with nothing installed. Paid subscribers get it, plus probe.py for when you want the number from a real crash, and the four implementations above. Upgrade and find yours.How to Pick
Four questions, in this order.
- Is your agent already a graph?
- Must its state commit in the same transaction as your own data?
- Must step payloads stay inside your network?
- Do you have anyone free to run a cluster?
The tree below shows where each answer sends you.
Most readers stop at the first question. If your agent is already a graph and no shared rate limit has to be rationed between tenants, attach a checkpointer, set durability to
sync, and you are done in an afternoon.Question three is the only one you do not get to decide. Compliance either lets step payloads leave your network or it does not, and a yes there removes Inngest before you have compared a single feature. If payloads can leave your network, you can have Inngest running this afternoon, with no queue and no cluster.
The table below is the same four decisions as a reference you can screenshot.
Note: read the third column first. “Best for” is where every comparison spends its words and it is the column that changes least, because all four are good at the thing they exist to do. What differs is who each one locks out, and that column is the one your compliance team, whoever owns the database, and your headcount decide for you. The setup times measure one engineer getting a first durable workflow running in staging. A migration takes longer.
Where Teams Get This Wrong
They pick a durable layer and keep the same step boundaries. Wrapping your whole agent in one
step.run() is a queue with extra steps. The saving comes from the granularity: one step per LLM call means a crash re-pays for one call, and one step for the whole loop means it re-pays for all forty.They never test recovery. Nobody has killed the workflow mid-run, so the step boundaries in your repo are still a guess about what the tool saves. I killed all four while writing this, and the tables above are what came back. Your own step boundaries are still a guess until you do the same, which is exactly why the probe ships with them. You have the same gap in the retry logic and the eval suite, which is what Why AI Agents Keep Failing in Production is about. The failure is always the same shape: something expensive was outside a boundary, and nobody found out until the bill.
They pick for month one and pay in month nine. A workflow that started in March has to finish under the code you ship in August, and by then you have renamed two steps and reordered a branch. Temporal makes you declare that change and version the workflow around it. Inngest keys memoization on the step name, so a rename orphans every in-flight run. LangGraph and DBOS leave the question to you. Nobody asks this in the evaluation, and everybody meets it eventually. A resumed run that answers differently is an eval failure before it is a durability failure, and What is an Eval? covers how you catch it.
They confuse durability with idempotency. Durable execution promises never to re-run a step that reported success. It promises nothing about a step that charged a card and then timed out before reporting anything. You still solve that one.
All four resume a crashed function, and none of them will be the reason your agent works. Watch what each one adds around the resumption instead. That is what you will actually be living with in year two.
The One Thing to Remember
Rule out a graph, rule out your own Postgres, rule out a cluster, and Inngest is what is left. Inngest also paid to be here. Both of those are true at once.
If you are starting an agent today and you want one answer rather than four: build it as a graph, attach a Postgres checkpointer, set durability to
sync, and move on. That covers the crash, it costs an afternoon, and it is not the tool that paid to be here. Come back to the other three when you can name the specific thing the checkpointer failed to do.The tool is not what decides whether a crash costs you thirty dollars or seventy-five cents. Your step boundaries decide that, and you draw those yourself in either case. Go find out what it costs you
What killed your last long-running agent, and did it resume or restart? Tell me in the comments.Leave a comment
Where to Next?
Go Deeper: The Open-Source Agent Toolkit in 2026 What you build the agent out of, before you decide what keeps it alive.
Go Simpler: What is Agent Memory? The other per-request state you carry across steps, and why a crash treats it the same way.
Related: Single-Agent Patterns The loop shapes these four wrap, and which ones have the most boundaries to draw.
FAQ
Is durable execution the same as a job queue?
No. It saves progress inside one unit of work. Your existing queue retries the whole unit, so the agent that died on step 38 buys all 38 of them a second time. You can build resumption on top of a queue, and that build is the project all four of these tools replace.
Do I need this if my agent finishes in ten seconds?
Probably not. The value scales with how much work sits between the start and the crash. A ten-second agent that fails costs you ten seconds. Wrap it in a retry and move on. The threshold is roughly where a full restart starts costing real money or real minutes.
Can I add durable execution to an agent I already wrote?
Yes, and the work is drawing step boundaries rather than rewriting logic. You wrap each expensive call in whatever the tool’s step primitive is. Temporal is the exception, because determinism is a constraint on the workflow function itself, so an agent that reads the clock or calls an API inline needs that code moved into Activities first.
1 - Durable execution, LangChain docs
2 - How Inngest functions are executed, Inngest docs
3 - DBOS Architecture, DBOS docs
4 - Workflow definition, Temporal docs