The chat panel paces streamed tokens into a typing effect. I drove both chat backends against a real matter, recorded the token-arrival traces, and simulated four pacing modes over them. The winner isn’t the one with the lowest lag — it’s the one that’s smooth and timely at once.
Ship Adaptive pacing, tuned to track ~one chunk-gap behind the head. Pacing streamed chat is a jitter-buffer problem, and it has two axes that trade off: lag (how far behind the live stream the reader is) and smoothness (whether the text flows or freezes-and-bursts). No fixed rate can win both.
Today’s production pacing (Default) is smooth but runs ~10 seconds behind on a real matter summary and keeps typing for ~10s after the model has finished. Turning pacing up (Fast) fixes the lag but reveals in bursts — ~460ms freezes between chunks. Turning it Off is instant but shows raw ~66-character jumps. Adaptive is the only mode that is both close to the head and smooth; tuning its horizon down from the shipped value roughly halves its lag (~1.4s → ~0.7s) with no loss of smoothness.
This replays an actual recorded token-arrival trace for the answer to “give me a summary of this matter” (matter mat_3irqt3v9gu5rk6w4, local dev). Same arrivals, four reveal strategies. The bar under each pane is how far behind the live stream it is; a red flash is a freeze; the counter tallies freezes, and when a pane finishes it shows its total time to reveal everything (compare against “tokens stopped at…” in the corner). Watch Off jump in ~66-char lumps, Fast burst-then-freeze, Default fall hopelessly behind and keep typing long after the tokens stop, and Adaptive glide just behind the head and finish moments after.
The first surprise from the traces: for a matter summary, the model spends most of the wall-clock not writing. On the Python backend the first answer character lands at ~5.6s (tool calls and retrieval first); on the agent platform, at ~1.7s. Then the answer body arrives as a burst of ~66-character chunks about every ~400ms, usually with one multi-second pause partway through where the model runs another tool.
The second surprise corrects something I believed going in. Watching raw network flushes, the agent platform looked ~12× finer-grained than Python. But most of the agent platform’s flushes carry zero text — they are status and reasoning events during the tool phase. Counting only text-bearing flushes, the two backends deliver the answer at almost the same cadence:
| Backend (answer text only) | first text | chunk gap p50 | p90 | mean chunk | burst rate |
|---|---|---|---|---|---|
| real Lawrence (Python) | ~5.6s | 379 ms | 631 ms | 69 ch | ~200 ch/s |
| lawrence-mini (agent platform) | ~1.7s | 415 ms | 543 ms | 66 ch | ~175 ch/s |
So the agent platform “feels faster” because it starts typing sooner and shows more intermediate activity, not because the answer streams in finer pieces. For the pacer, the job is the same on both: smooth out ~66-character lumps arriving every ~400ms, during bursts that run at ~175–200 ch/s.
I simulated each pacing mode over all six recorded traces, faithful to the shipped algorithms. Two things matter and they pull against each other: peak lag (how far behind the head the reader ever falls, and how long the pane keeps typing after the model stops) and smoothness (the longest freeze during the sustained typing burst, measured with the tool pauses excluded since no mode can reveal during those).
| Mode | peak lag | typing after model done | longest freeze (burst) | character |
|---|---|---|---|---|
Off | 0 | ~30 ms | ~570 ms | instant, but raw ~66-char jumps |
Default prod today | ~10–11s | ~10s | ~250 ms | smooth, hopelessly behind |
Fast | ~370 ms | ~80 ms | ~460 ms | timely, but burst-then-freeze |
Adaptive as shipped, H=800 | ~1.4s | ~440 ms | ~230 ms | smooth & timely |
Adaptive tuned, H≈300 ← recommend | ~0.7s | ~260 ms | ~270 ms | smooth & timely, tighter |
Averages across 3 runs per backend, 6 traces total. “Peak lag” and “typing after done” are the worse of the two backends; “longest freeze” is the sustained-burst figure.
Why no fixed rate can win. The answer arrives in ~66-character lumps every ~400ms. To reveal it smoothly you must always have characters in hand to dribble out — which means staying at least one lump-interval (~400ms) behind. Reveal any faster and you catch the head, run dry, and freeze until the next lump: that is Fast (and Off is the limit case, revealing each lump in a single frame). Reveal slower than the burst rate and the backlog grows without bound until the model stops: that is Default, which at ~80 ch/s simply cannot keep up with a ~200 ch/s burst and ends up ~10s behind. Adaptive sidesteps the dilemma by setting its reveal rate from the backlog each tick, so it self-parks a fixed time behind the head — smooth by construction, and as timely as that target allows.
Adaptive shipped with a horizon of H = clamp(2 × EMA(gap), 150, 800)ms. Given ~400ms chunk gaps, that pins H at its 800ms ceiling and parks the reader ~1.4s behind — smoother than necessary at the cost of lag. Tightening the target to about one chunk-gap is strictly better on this data:
| Adaptive horizon | peak lag (Python / mini) | typing after done | longest freeze | frames frozen |
|---|---|---|---|---|
H=800, 2×EMA shipped | 1400 / 1374 ms | ~450 ms | 83 / 383 ms | 67 / 68% |
H=450, 1.2×EMA | 984 / 990 ms | ~310 ms | 83 / 383 ms | 67 / 68% |
H=300, 1×EMA, 33ms tick ← recommend | 757 / 670 ms | ~260 ms | 139 / 406 ms | 50 / 52% |
Halving the horizon halves the lag (~1.4s → ~0.7s) and, with a faster tick, actually reduces the fraction of frozen frames (67% → 50%). There is a floor: push H below one chunk-gap and Adaptive starts to run dry between chunks and reintroduces the Fast-style freeze. ~one chunk-gap (~300–400ms) is the sweet spot — the literal answer to “as fast as possible without hitting the head.”
H_MAX ≈ 350, multiplier 1×EMA, and tick faster (rAF or ~33ms)#12530; this is a constants change plus the tick sourceclaude-sonnet-4-6; the answer-text cadence is close enough (~380 vs ~415ms p50) that one tuned Adaptive setting serves both. Re-check if a backend’s model or streaming path changes.