First dry runs in dev: what we measured

Companion docs: the coverage loop, the v2 boundary mapping (where the 714-matter dump comes from), the LEAP boundary mapping.

The two runs

The same 714-matter dump ran clean locally last week, so run 2's failures are environmental. The 38-second work figure comes from the cell timestamps: 30 gaps over one second between consecutive cell updates account for 1,119 of the run's 1,157 seconds.

Where the 19 minutes went

The run is a chain of Inngest function invocations, one per batch of ~100 units, each ending by emitting the trigger for the next. Between invocations the service sits idle waiting for Inngest to dispatch. Measured on run 2:

• ~30 batches, average dispatch gap 37s, steady state 20–50s.
• One 662s gap immediately after batch 1, during the window when a deploy had just replaced the task and the old worker was still draining.
• The gaps are not retry backoff: the 302 failures all happened in the last minute of the run.

So the cost model is wall time ≈ batches × dispatch gap + rows × 6ms, and the first term is everything. Two consequences:

1. CPU sizing is irrelevant right now. The task computed for 38 seconds. Doubling vCPUs saves 19 of them.
2. Batch size divides the dominant term directly. At batchSize=1000 this run is ~3 batches and finishes in 2–3 minutes.

The open question is why dispatch takes 20–50s at all; Inngest normally dispatches in about a second. The likely suspect is the function's concurrency: 1 cap, which Inngest enforces by polling a queue rather than pushing. One instrumented run answers it, and the answer decides how hard to lean on batch size.

Projections for v2 and LEAP

At today's settings (batch 100, ~37s gaps, ~200 rows per batch at this firm's density):

Density is the error bar: the dev firm averages 8.5 rows per case with few files or messages. Heavier firms scale these numbers linearly. Real runs also write to four databases, which dry runs skip; expect some increase, on the order of the work term, not the idle term.

Dry runs are only correct in one process

Run 2's failure signature: matter_participant references Identity 'psn_…', which no run has imported for this firm, on 302 matters, referencing identities the same run had read and registered (419 of 419, balanced).

Cross-references resolve through an in-memory cache, falling back to the persisted ImportedRecord registry on a miss. Real runs write registry rows, so the fallback works from any process. Dry runs persist nothing, so resolution silently requires every batch to run in the process that registered the earlier passes. Run 2 started during a deploy cutover with two workers connected, Inngest spread batches across both, and every batch on the second process lost its lookups.

Cutover go/no-go will rest on a clean rehearsal. A rehearsal whose verdict changes when a deploy lands mid-run cannot carry that weight.

Two smaller reliability items from the same runs:

• Both runs sat minutes in PENDING looking identical to a dead run. The run row has a lastProgressAt column nothing writes. Writing it per batch, plus a "queued" state in the UI, turns "is it dead?" into a glance.
• Run creation streams the whole dump to verify checksums before the run row exists, inside the tRPC call the admin screen waits on. Fine at 2.3MB, a multi-minute hang at a heavyweight firm. The verify belongs inside the run.

Files

Both runs imported file rows only: 1,531 matter_file rows with no bytes behind them, by design. File binaries are a separate migration framework, built once for v2 and LEAP/WS both. Production real runs stay refused until it exists.

Facts the framework starts from, established this week: the dump carries no bytes (payload_uri points at v2's live case-files bucket); the destination is matter-service's documents bucket, cross-account, reached today by static IAM keys; nothing on the path is malware-scanned, so the prod scan gate would reject every file; and each placed object would trigger matter-service's scan-and-preview pipeline, ~1,500 Lambda invocations for even this small firm.

The dump was complete, and now we can prove it

Run 2 prompted the first source-vs-dump reconciliation. v2 dev holds 715 cases for this firm; the dump holds 714; the difference is one case created after the export snapshot. Getting there required writing down what "a firm's cases" means in v2: the union of four association paths (team-member parties, external parties, employee person-parties, solicitor-owned). The dbt canonical models should be checked against the same rule, and exports should carry source-side row counts so completeness is a check rather than an investigation.

Next work, in order

1. Persisted dry-run registry. Rehearsals correct under deploys and multiple workers.
2. Measure the dispatch gap. One instrumented run tells us whether 37s is the concurrency cap, the dev Inngest tier, or something else.
3. Batch-size experiment. Same 714-matter dump at batch 500 and 1000; turns the projections into a curve.
4. lastProgressAt heartbeat and a queued state.
5. Async checksum verification at run creation. Needed before the first heavyweight firm.
6. Files migration framework plan (v2 + LEAP/WS), starting from the facts above.
7. Export-side reconciliation: source counts in the manifest; dbt firm-case rule checked against v2's four-path union.

Shipped alongside the runs: Axiom telemetry for the import task (run 2's diagnosis came entirely from it), notes on runs for operator context, and a copy button on the run JSON.