The content service, end‑to‑end

A quick way to hold the whole service in your head: there is exactly one thing that is authoritative, the stream of collaborative edits, and everything a caller can ask for is a projection of it. Start there, because every other design decision falls out of it.

Here is a single matter document, a short client letter. Internally it lives as one Yjs document (a ProseMirror tree). Click through the formats a caller can request: each one is the same content, transformed on the fly by a different chain of libraries. Watch the transform chain and the libraries change underneath. This is the heart of the service, so it goes first.

You asked about document-service and content-service together, so start with the honest headline: there is only one service now. document-service was deleted in PR #11249 ("Remove the unused document-service", June 2026). What you see on disk under services/document-service/ is the wreckage: a dist/ build folder, a .turbo log, and two generated Prisma artifacts (schema.dbml, enums/index.ts). None of it is tracked by git anymore. There is no src/, no package.json, no schema.prisma.

Its history is short. It was carved out of identity-service in January 2026 to own firm precedent templates: upload a DOCX, store it in S3, convert it, preview it, post-process it with an LLM. It never stabilised. The removal PR notes it "was already pointing at the identity service URL as a placeholder and was clearly non-functional". Meanwhile content-service was created fresh in February 2026 (#8153) on a completely separate track, matured quickly, and made the template-owning service redundant. This was a supersession by deletion, not a code merge: no document-service code or migrations moved across. content-service rebuilt the idea from scratch, more generally.

The contrast is the clearest way to see why content-service is shaped the way it is. The old model baked the domain (a firm precedent, in a jurisdiction, in a legal area) straight into the table. The new model pushes all of that meaning up into the calling service and keeps only a generic, versioned, collaborative body.

Ten tables in one Postgres database (content_svc), in three loosely-coupled clusters: a file cluster, a content cluster (the interesting one), and a template cluster layered on top of content. Click any entity to see its fields and how it links to the rest. The composite relations are the subtle part: a snapshot is identified by (id, contentId), so a version can only point at a snapshot that belongs to the same content. That is enforced in the schema, not in application code.

This is the idea everything else hangs off. A document's body is a Yjs CRDT: a conflict-free replicated data type that lets many editors (a lawyer, the editor's autosave, Lawrence) mutate the same document and converge. content-service never stores "the current document" as a single blob it overwrites. It stores an append-only log of update bytes (ContentUpdate), plus periodic materialised snapshots (ContentSnapshot) so it does not have to replay the entire log on every read.

Reading "the latest" means: take the most recent snapshot, then replay every update that landed after it. Reading a version means: use that version's snapshot directly, no replay.

Two integers keep concurrent writers honest. currentEpoch is an optimistic-concurrency guard: a writer sends the expectedEpoch it read, and if it no longer matches, the write comes back stale and the client must re-read. The batchId is an idempotency key: replaying the same batch returns exists rather than duplicating it. Play with it below.

Versions, snapshots and the auto-milestone

A ContentVersion is the user-visible layer on top of all this: a label ("Automatic milestone", or whatever a lawyer types) pinned to one immutable snapshot. Versions come from three places. A caller asks for one explicitly. A new version is created from a source (a fresh DOCX upload, some Markdown, another document). Or the service makes one on its own.

That last path is the auto-milestone. Every persisted update fires a content-service/content.updatePersisted event. An Inngest function debounces those per document (120s, capped at one hour), and once a document has had edits and its latest version is at least 15 minutes old, it checkpoints a new "Automatic milestone" version with a null author. It is the safety net that means history exists even when nobody clicks save.

The explorer above showed the outputs. Here is the machinery. The canonical in-memory shape is a ProseMirror tree living inside a Yjs document, in the Yjs XmlFragment named "content" (the constant CONTENT_ROOT). The binding between ProseMirror and Yjs is @tiptap/y-tiptap (TipTap's successor to y-prosemirror): prosemirrorJSONToYDoc on the way in, yXmlFragmentToProseMirrorRootNode on the way out. The ProseMirror schema itself is built once from getOoxmlExtensions(), a TipTap extension set shared by every conversion, so Markdown, HTML, DOCX and the editor all agree on what a valid document is.

Everything inbound funnels through one hub, createContentFromSource: plain text, Markdown, HTML, ProseMirror JSON, a raw Yjs snapshot, another document, or a template all become ProseMirror JSON, which becomes a Yjs snapshot. Everything outbound funnels through getContentData, which reconstructs the Yjs state and projects it. The libraries doing the work:

Why a DOCX round-trips faithfully: the shell strategy

A Word document is far more than its text: styles, numbering, a theme, headers and footers, fonts. If content-service parsed a DOCX into ProseMirror and later regenerated it from the ProseMirror alone, all of that would be lost. So on import it keeps the original. The bytes are parsed into ProseMirror, and the document's OOXML metadata is serialised into a ContentShell linked to the source file. On export, that metadata is re-applied and the content is patched back into the original .docx as a shell buffer. The text is yours; the chrome is the file's.

The file domain is the only part of content-service that talks to S3, and it exists so that "upload a file, scan it, mark it ready" lives in one place instead of being rebuilt in every domain that needs an attachment. A file starts as a PENDING row with a presigned upload, and the bytes go straight to S3 from the client, never through the service. The S3 key encodes the scope, so storage layout follows ownership:

• files/<scope>/_external/<fileId>.data  — a user upload
• files/<scope>/_internal/<fileId>.data  — platform-generated
• files/<scope>/_derived/<fileId>/<artifact>  — a derived asset

where <scope> is a typed path like matter/mat_123 or firm/fir_1/matter/mat_2. The presigned POST caps uploads at 200 MiB and expires in 120 seconds. Only original .data objects take part in the lifecycle below.

The lifecycle hinges on one distinction. INTERNAL files are trusted (the platform made them) and jump straight to READY. EXTERNAL files (a human uploaded them) go to UPLOADED, trigger a GuardDuty malware scan, and only reach READY after the scan comes back clean and the object carries the verifying tag. The scan result arrives the long way round: GuardDuty publishes to SNS, which hits webhook-api, which emits an Inngest event back into content-service.

Everything that cannot happen inside a request happens through Inngest. There is a subtlety worth flagging up front: content-service has a @lawhive/framework event-store wired in (the Event table), but its events router is empty, so nothing writes to it today. Every content-service/* event you will see is an Inngest event, sent with inngest.send() and consumed by an Inngest function. The framework store is scaffolding for a future that has not arrived.

A second thing to notice: every content-service/* event is produced and consumed entirely within content-service. No other service listens to them. The only cross-service coupling is at the webhook/s3/* layer, where content-service and matter-service both listen to the same S3 webhook events but filter on different buckets. content-service is more self-contained than it first looks.

Read the catalog top-to-bottom and one worked flow appears, the template DOCX import. createTemplate makes a backing Content, an uploadable File aimed at it (targetContentId), and a Template in PROCESSING. The lawyer uploads a DOCX. The file becomes READY, which fires file.ready, which runs createVersionFromFile (parse, build the ContentShell, create a version), which fires versionFromFile.ready, which flips the template to ACTIVE. If anything stalls, the 5-minute cron sweeps it to FAILED. Four functions, one event chain, no orchestrator.

Because content-service holds no domain meaning, it cannot authorize on domain rules. Instead every File, Content and Template carries a scope: a typed, ordered path serialised to a string, such as matter/mat_123, firm/fir_1/matter/mat_2, or identity/pers_9. Scope does three jobs at once: it is the authorization key, the prefix you list by, and the S3 key layout. One concept, three uses.

Four middleware procedures enforce it. protectedScopeProcedure guards create and list operations against the input scope. protectedContentProcedure, protectedTemplateProcedure and protectedFileProcedure load the existing entity, check scope access, and return a masked NOT_FOUND when the caller is not allowed (so absence and forbidden look identical from outside). The service itself trusts an internal JWT; identity propagation across service boundaries is what makes the matter-scope checks meaningful.

And who actually calls it? content-service sits two hops below the UI. The primary consumer is matter-service, which owns what a "matter document" means and delegates the content mechanics down. The chain for an ordinary editor action:

Lawrence never imports content-service. It reaches documents through a Virtual File System that lawrence-api exposes, addressing everything as paths like documents://<matter>/<doc>. On the agent side, the Python platform-vfs package turns the tools read, stat, list, create and edit into KMS-signed HTTP calls. On the platform side, an adapter registry maps each protocol to a backend. The path for a document is: agent → platform-vfs → lawrence-api VFS → matter-service → content-service.

Read is Markdown, edit is Yjs

The two directions are deliberately asymmetric. A plain read on a document returns Markdown by default (the agent can also ask for ProseMirror JSON, a single section, or a block map). But an edit does not write Markdown back. It is compiled into Yjs updates. The agent sends high-level commands ("in block b_a2, replace 'Client' with 'Party'"); lawrence-api fetches the current ProseMirror and Yjs state, checks the content_version_id the agent saw still matches, applies each command as a diffSuggestion mark (a tracked change the lawyer reviews, not a silent overwrite), encodes the mutated Y.Doc, and submits it through matter-service to content.applyContentYjsUpdate. The same epoch CAS from the storage section guards it: a stale result triggers one re-read and retry.

The trace below is that round-trip, with the real request and response shapes. The write verb, by contrast, is disabled for documents: editing always goes through edit so it lands as reviewable suggestions.

Here is the most common misconception about this corner of the platform, worth stating plainly because the schema invites it. content-service's File has an ingestionStatus column (NOT_REQUESTED / PENDING / SUCCESS / FAILED / UNSUPPORTED). It is tempting to assume that is how an uploaded document gets chunked, embedded and indexed for Lawrence's search tools. It is not. That field is a dormant scaffold: nothing writes to it (the updater has zero non-test callers), and it never leaves NOT_REQUESTED.

There are two different File models on two parallel tracks. content-service's File is the editable track (the content bucket, Yjs documents). The ingestion-and-retrieval track runs entirely through matter-service's own File (the matter-documents bucket), and its status lives in matter-service columns named aiIngestionV2Status / aiIngestionV3Status. They do not meet today.

The real loop, end to end, is a different set of services entirely. It is worth tracing once so you never confuse it with content-service again:

A field guide is only useful if it tells you which parts of the map are real. content-service has a handful of columns and helpers that look load-bearing but are not yet wired, plus a couple of docs that have drifted from the code. Here is the honest state, so you do not waste an afternoon chasing a pipeline that does not exist.

The bits you will want again: id prefixes (every id is self-describing), the procedure surface by domain, and the runtime facts. At the end there is a button that copies a distilled summary of this whole page as Markdown, so you can paste the mental model straight into Claude Code or a ticket.