Data Migration Engine · Sprint 1 · 6 – 19 August 2026

Sprint 1

The migration engine takes shape

First sprint · Final
Lawhive

Why this squad exists

Two firms are waiting to move

Lawhive Legal
on platform v2
37,958
cases
1.65m
files
1.8 TB
of documents
Woodstock
on LEAP, a third-party system
27,005
matters, back to 2014
?
documents: LEAP holds them, count unknown

Both firms need their history in platform v3, safely and verifiably. The engine is built once and is source-agnostic, so every firm we migrate after these two reuses the same pipeline and the same import service.

Lawhive

How we are building it

Four stages, one contract

Data pipeline · BigQuery · dbt · Dagster Platform v3
Stage 1 · Extract
Source lands raw
v2 replicates continuously; LEAP arrives as report exports, with an API route for documents.
Stage 2 · Stage & map
dbt reshapes rows
One model per boundary table, per source. Source quirks are resolved here, and recorded.
Stage 3 · Package
Dagster writes the packet
One sorted JSONL file per table, plus a manifest, into regulated S3.
Stage 4 · Load
Import service writes v3
Validates every row, writes with provenance, and can tear a run down to zero.
The contract: 17 boundary schemas
Zod schemas in platform-v3 are the source of truth; a generator emits matching dbt contracts into the warehouse, so the two halves cannot drift apart silently.

Why it is shaped this way

Each half is built and tested on its own; they only meet at the packet Any source that can produce the packet can be migrated Every imported row records where it came from, so a run can be verified and undone
Lawhive

Sprint 1

Agree the architecture, land the import service, and start extraction and mapping for both source firms.

39 tickets done · git-verifiedcounted from trunk merges, not from Linear. The team does not point tickets, so counts, not points.
27
PRs landed across 3 repos
14
design docs published
11
tickets in review or in progress

Milestones · progress toward the first live migration

Staging & infrastructure
50%
Platform v3 readiness
50%
Data contracts / canonical model
33%
Import-service readiness
30%
Files migration mapping
25%
End-to-end load of a matter
17%

First sprint, so there is no velocity trend yet. Milestones from the Linear project, after the close-of-sprint status sync; the project targets 20 October for the full Lawhive Legal load.

Lawhive

Load · stage 4

The import service is on trunk

Merged to develop
✓ Boundary catalogue all 17 schemas ✓ Cursors & walk streams the packet in order ✓ Transactional writes unit + provenance together ✓ Report & teardown per-cell grid, undo to zero ✓ Control API & runbook create, cancel, resume, inspect
  • Proven end to end before merging: the full 714-matter dev firm imported, torn down to zero, and imported again 10,184 rows, zero failures, and the dry run produced a result identical to the real run.
  • Every imported row gets a provenance record in the same transaction, so a run can be audited, verified against the manifest, and undone
  • Security review passed: real runs are gated on super-admin and refused in production until deliberately enabled there

Before the first real run: deployment wiring (nothing ships the service's image yet) and a real file store (imported file rows carry no bytes yet). Both are named next-sprint work.

Lawhive

The contract

Both sides now speak the same schemas

Landed · sync in review
  • The 17 boundary schemas merged in platform-v3: matters, participants, contacts, files, notes, tasks, key dates, messages, calls
  • A generator emits matching dbt contracts into the warehouse, so a pipeline model that drifts from the contract fails its build instead of failing at load time
  • The receiving dbt package landed in data-dbt; the GitHub Action that keeps the two repos in sync automatically is in review

Alongside the contract, v3 itself was made ready to receive: matter references are now per-firm strings across the whole product, so an imported matter keeps the reference its firm has always cited, and every service database gained the provenance table.

Lawhive

Extract & map · stages 1–2

Both sources are moving

In progress
  • Lawhive Legal (v2): the mapping is written and proven on a real matter across all 17 tables 28% of the book maps onto the catalogue today, another 17% with agreed additions; 52% is money data, parked for a later, separately signed-off wave.
  • The first v2 staged slice is in review: staging models, mapping seeds, the catalogue as a dbt package, and the packet exporter
  • Woodstock (LEAP): the full book is profiled and the mapping rehearsed over all 27,198 matters Every source row is resolved, mapped, or excluded with a written reason. LEAP's 58 statuses collapse to v3's four lifecycle states, keeping the original as a firm field.

The Woodstock critical path is LEAP data access: the automated feed has been dead since March, a client-id export has been requested, and documents need the LEAP API route. These decisions sit with LEAP, not with us.

Lawhive

Migration projects · Cameron, Aadam, Sally, Kha-Ai

The operational track is stood up

Programme scaffolding live
  • The programme home is live on Notion: goals, the two pillars, and one project page per source firm A migration is run as business engagement, not only engineering: data-quality agreements, mappings signed by people who use the data, and named sign-offs.
  • Every migration runs the same five-phase template, discovery through cutover, with ten gates, each carrying an exit condition and a named deliverable
  • The first project, Lawhive Legal UK, is scoped and staffed: RACI agreed, a v2 to v3 data dictionary, a pre-mortem, and testing plans that define done Done for the first slice: one matter end to end in dev, every automated check green, run twice with the same result, and a lawyer can open the migrated matter.

The rule that de-risks Woodstock: discovery does not close until extraction is proven by a spike, and the LEAP contract is never terminated until the extract is verified. LEAP allows only a 30-day window from termination.

Lawhive

Around the edges

Also this sprint

Regulated data handling
A dedicated S3 bucket for migration dumps, with a written retention and protection policy, backup registration, and scoped Dagster access.
PII tagged at the source
One staff-PII tag taxonomy in the warehouse, so sensitive columns are governed in one place.
Matter references, everywhere
Search, notifications, PDFs and sorting all handle the new string references, for native and imported matters alike.
Lawhive

What's next

Next sprint

Pending sprint planning
First slice end to end
One v2 matter through all four stages into v3 on dev. The first time the pipeline and the import service meet on a real environment.
Pipeline
Land the in-review dbt stack, staging models for every in-scope entity, and the packet exporter running inside Dagster.
Import service
Deployment wiring so the service actually ships, and a real file store so imported files carry their bytes.
Verification
Reconciliation models so nothing is silently dropped, and the coverage loop over the full Lawhive Legal book.
Lawhive

Thank you

Full report → Sprint 1, Data Migration Engine Design docs → firm-migrations Linear → Build & Test Migration Engine
Lawhive