An AI Reply That Survives a Restart

Ask Suma, the assistant in Sumalos, to "invoice the three workshops from last week and email them", and the answer takes a while. The model thinks, searches the contacts, drafts three invoices, writes a summary, asks before sending. Twenty seconds is normal. In those twenty seconds the person switches tabs and the browser reloads the page, or the train goes into a tunnel, or I deploy core-api.

In the usual setup, the answer lives inside the HTTP request that asked for it. The model streams into the response, the response goes to the browser, and when the connection dies the generation dies with it. What the person sees is half a sentence and a spinner that never resolves. What the system has is worse: two invoices drafted, the third not, and no record of where the model was.

So the question this post answers: how does an AI reply survive the connection, the tab and the process that started it, and pick up exactly where it was, without showing a word twice or running a tool twice?

The reply is not the request

The first decision does most of the work. A reply in Sumalos doesn't belong to the connection that asked for it. chat.send checks the session, the organization and the limits, inserts one row into a table called ai_runs, and returns two ids. That's all the request does. The answer is generated by a producer in the background, on the server, whether or not anybody is reading it.

The browser then opens a second connection, a server-sent events stream, to follow that reply. Any instance can serve it. If it drops, the browser opens another one.

The split between the two stores is the rule I kept coming back to: Postgres holds what matters, Redis only coordinates. If Redis disappeared entirely, the cost would be a few seconds of live delivery and nothing else.

A finished conversation costs nothing in Redis. A reply in flight costs about 300 bytes (the lease plus the control subscription), so 300,000 replies at once would be roughly 90 MB.

Every connection starts from the beginning

The obvious way to resume a stream is event ids: the browser says "I got up to event 412" and the server sends 413 onwards. That needs a buffer of recent events per reply, somewhere every instance can read, kept long enough for slow reconnects. I didn't want that buffer.

Instead, every connection starts with the whole reply so far. A reader subscribes to the reply's channel, asks the producer for a snapshot over the control channel, and the producer answers with every chunk it has produced, consecutive text deltas joined. After the snapshot come the live batches, numbered, in order:

for (const e of queue.splice(0)) {
  if (e.t === "snapshot") continue;
  if (e.fence !== fence) return void (yield { resync: true });
  if (e.seq <= seq) continue;
  if (e.t === "chunks") {
    if (e.seq !== seq + 1) return void (yield { resync: true });
    seq = e.seq;
    quietSince = Date.now();
    yield { chunks: e.chunks };

Redis Pub/Sub is fire and forget, so a batch can get lost. The reader doesn't try to repair it. A gap in seq, or a batch from a different producer, ends the connection with resync, and the browser reconnects and gets a fresh snapshot.

What makes this cheap on the browser side is a flag in the AI SDK. Its chat transport can declare that a resumed stream is a replay, and then the SDK rebuilds the message from scratch on each reconnection instead of appending to what it had:

return {
  resumeStreamIsReplay: true,
  async sendMessages({ chatId, messages, abortSignal, trigger }) {

So reloading the tab, losing the network, opening the same conversation on a second device, all become the same operation: open a connection, receive the reply so far, keep following. A dropped connection fails as a plain disconnect, the title bar says "Reconectando…" ("reconnecting"), and the browser retries with backoff from half a second to ten, or at once when the network comes back.

When the producer dies

Following a reply that is still being written is the easy half. The hard half is the producer itself going away: a deploy, a crash, a bun --watch reload in development. The reply is in the middle of a sentence, or worse, in the middle of a tool call.

Two things have to be true. Some other instance has to notice and take the reply over. And the old producer, if it wasn't dead but only paused (a long garbage collection, a frozen container), must not come back and keep writing next to the new one.

The first is a lease. The producer holds run:<id>:lease in Redis, set with an expiry of 15 seconds and renewed by a heartbeat every 5. The second is a fencing token, the pattern Martin Kleppmann describes for exactly this problem: a lock with an expiry can't stop a paused process from waking up and acting on a lock it no longer holds, but a number that only goes up can. The token lives in Postgres, in the fence column, and taking a reply over is one atomic update:

async claim(id, fence) {
  const [row] = await db
    .update(aiRuns)
    .set({ fence: sql`${aiRuns.fence} + 1`, attempts: sql`${aiRuns.attempts} + 1`, updatedAt: new Date() })
    .where(and(eq(aiRuns.id, id), eq(aiRuns.fence, fence), eq(aiRuns.status, "running")))
    .returning({ fence: aiRuns.fence });
  return row?.fence ?? null;
},

If two instances try at once, one update matches and the other finds no row. Every write the producer makes after that carries its token, where r.fence = v.fence, so a producer that wakes up with the old token finds nothing to update. Its next heartbeat finds the lease held by someone else, and it stops itself.

Who notices? Three paths, and any of them is enough:

• A reader that gets no answer: it asked for a snapshot, waited 1.5 seconds, and found no lease. It takes the reply over itself and keeps asking; the snapshot now comes from the new producer.
• The sweep: every instance, half a second after it starts and every five seconds after that, pages through the running replies and fetches their leases in one MGET. A reply whose lease has been missing for a whole lease period is taken. A lease from an earlier boot of the same host and process id is dead at once, which is what a bun --watch reload looks like.
• The handover: on SIGTERM, core-api stops taking replies, checkpoints its own, releases their leases and publishes their ids. Another instance takes them without waiting for anything to expire.

The lease and the token cover each other's gaps in one more case. If Redis loses every key (a flush, a restart), a live producer sees its lease missing at the next heartbeat, checks Postgres that the token is still its own, and takes the lease back. The sweep only acts after a lease has been gone for a full period, so it doesn't race the producers getting theirs back.

Resume, don't repeat

Taking a reply over is only useful if the new producer can continue it, and that is where the checkpoint comes in. The producer writes its chunks to the ai_runs row after every tool result and every five seconds while text streams. A new producer reads the checkpoint, rebuilds the turn (the conversation as it was stored, the person's membership as it is now, the page they asked from, the model it was using so the prompt cache isn't lost), and then repairs the checkpoint so the model can carry on from it:

export function resumableChunks(chunks: Chunk[]): Chunk[] {
  const out = chunks.filter((c) => c.type !== "finish" && c.type !== "abort" && c.type !== "error");
  const open = new Map<string, string>();
  const tools = new Map<string, boolean>();
  for (const c of out) {
    if (c.type === "text-start" || c.type === "reasoning-start") open.set(String(c.id), c.type === "text-start" ? "text-end" : "reasoning-end");
    if (c.type === "text-end" || c.type === "reasoning-end") open.delete(String(c.id));
    if (c.type === "tool-input-start" || c.type === "tool-input-available") tools.set(String(c.toolCallId), tools.get(String(c.toolCallId)) ?? false);
    if (c.type === "tool-output-available" || c.type === "tool-output-error" || c.type === "tool-output-denied") tools.set(String(c.toolCallId), true);
  }
  for (const [id, end] of open) out.push({ type: end, id });
  for (const [toolCallId, answered] of tools) if (!answered) out.push({ type: "tool-output-error", toolCallId, errorText: "Se ha interrumpido antes de terminar." });

Two repairs. An open text part is closed, so the continuation starts a new part after it and never writes into the middle of a sentence. And a tool call that never got its result is marked as failed ("it was interrupted before it finished"), so the model sees it and decides again.

The model then gets everything it said and did as its own message, plus one line in the turn's context. In the original Spanish it tells the model that its previous answer was cut off by a technical problem, that the person has already seen what it wrote, and to carry on exactly where it stopped, without mentioning it, greeting again, repeating itself or redoing what is done. The new parts go into the same message, on the same channel. A browser that was watching sees a few seconds of "Reconectando…" and then the answer keeps going.

Deciding again is where a resumed reply could do harm. If the model was halfway through creating two contacts, it may ask for the same action again. That's safe because every action carries an idempotency key built from the message id: asked a second time, it returns what it already did, or runs only the steps that hadn't run, since each step stores its own result. How those keys and plans work belongs to the post on confirming agent actions. What I watched happen in a real restart was gentler than that: the reply was cut during an action creating two contacts, and the resumed model checked with a search and a query whether they existed before doing anything.

There is one thing this can't fix. A step that was halfway through running and hadn't stored its result yet, for example an email already handed to the provider, could run twice. Without help from the outside provider, nothing on my side can tell whether it went out.

A reply that keeps dying doesn't get resumed forever. After three takeovers in a row without progress (each new checkpoint resets the count, so a run of reloads in development doesn't use up the attempts of a reply that is moving), or fifteen minutes in total, it ends cleanly: what was said stays, with a notice that the reply was interrupted and a Retry button.

WhatsApp has no stream

On WhatsApp the agent sends the whole answer at the end, so there is nothing to follow live. The same table still tracks it: each incoming message gets an ai_runs row with channel = whatsapp and a lease renewed while the agent works. The WhatsApp agent is the only process holding the session, so recovery runs when it connects, over the WhatsApp replies still marked running whose lease is gone or belongs to an earlier boot.

The check in the middle is the one that keeps WhatsApp from double replies: the agent's own sent messages are stored too, so if it already answered in that chat after the message, the run is closed as done. Otherwise the message goes back through the normal pipeline and is answered once, with the same idempotency keys protecting its actions. A message that has failed three times gets one short apology asking the person to send it again.

What it costs

The Redis side is small by design. The Postgres side is where the writes go: about five or six per 20-second reply (the insert, a checkpoint per tool result and every five seconds of text, the end). Checkpoints from all the replies in one process are grouped into 100 ms windows and written as one UPDATE ... FROM (VALUES ...). The rows are narrow, no indexed column changes on a checkpoint, and the table has a fill factor of 70, so most updates can be heap-only tuples that don't touch the indexes.

The load test ran in development, against a remote Redis and Postgres through an SSH tunnel with roughly 45 ms and 100 ms of round trip:

Those numbers say more about the tunnel than about the design; with a local Redis the limit becomes CPU.

The tests that matter more are the ones that kill things. The streams suite, against a real Redis with a throwaway prefix, kills the producer mid-text and mid-tool, kills it repeatedly without progress, sends SIGTERM, brings back a zombie with an old token, makes two instances race for the same takeover, and deletes the lease under a live producer. The browser tests in Playwright reload mid-answer, open a second tab on the same reply, cut the network, and restart core-api during a real model answer: in all of them the final message is identical to the stored one and appears once. The restart shows "Reconectando…" for about five seconds.

The connection was never the point

Looking back, almost every piece follows from the first decision. Once the reply isn't the request, a reload is just a second reader. Once every reader starts from a snapshot, there is no event buffer and no duplicate text. Once the state is a row with a token, a dead process is just a row whose lease nobody renews, and taking it over is one UPDATE that only one instance can win.

The price is a write to Postgres every few seconds for every reply in flight, and a model that sometimes has to be told, in so many words, that it was interrupted and should carry on without saying so. For an assistant whose answers draft invoices and send emails, I'll take that over a spinner and half a sentence.