LawHive is acquiring law firms whose phones still need answering. This is the plan for a service that picks up every inbound call: a human in working hours, an AI agent after them, recorded and transcribed, and able to transfer or hang up on its own. It stays out of the call's audio entirely, and one provider pattern plugs in ElevenLabs and our own speech-to-speech gateway alike.
Mostafa Yassin · Engineering
PoC validatedDraft · 2 July 2026Telephony: TwilioStatus: for review
Incoming call
+44 20 7946 0000
In hours → human
After hours → AI agent
Watch a call move through it
An after-hours call handled by a self-hosted speech-to-speech agent, step by step. The telephony service registers the call, hands Twilio a token-guarded stream URL, and steps out of the audio. Every hop is a real webhook, register call, WebSocket, or Twilio REST call.
Call traceself-hosted gateway shown
The problem
When LawHive acquires a firm, its phone line does not stop ringing. Clients call about live matters, and a missed call is a lost instruction or a worried client left hanging. Staffing every line for every hour is not realistic across dozens of firms, and a plain voicemail throws away the intake.
Each firm needs the same thing: ring a real person in working hours, and after hours (or when nobody picks up) hand the caller to an AI agent that can take the intake, answer routine questions, escalate to a human, and end the call cleanly. Every call recorded, every call transcribed. And it has to work without our standing up a telephony stack we then operate.
The service picks the caller up and steps out of the audio: it decides who answers, and a provider holds the call.
One provider pattern
Twilio owns the telephony: the numbers, the media, the recording. The service runs HTTP endpoints only and never holds the call's audio. Per firm it stores just an opaque agent reference (a provider and an agent ID, like an ElevenLabs agent ID); the agent's own configuration lives with the provider that runs it.
On an inbound call the service resolves the firm to a provider and agent ID, makes an authenticated register call to that provider, gets back a media stream URL carrying a short-lived token, and returns <Connect><Stream> to Twilio. The media flows Twilio to provider, never through us. The provider runs the agent and calls back for the two things only we can do: act on the Twilio call, and store the transcript. ElevenLabs and our own gateway are the same shape.
Managed runtime
ElevenLabs
Caller→Twilio→ElevenLabs
Their runtime holds the media and runs the agent. We register the call and hand Twilio their token URL. Almost nothing to operate.
Self-hosted runtime
Our speech-to-speech gateway
Caller→Twilio→our gateway→model
Our runtime for raw models (OpenAI Realtime, Grok, Gemini Live). Same seam: register, token URL, Bearer callbacks. It is the warm, stateful piece; the telephony service stays deployable.
media never touches the telephony service · both use register + token URL + Bearer callbacks
The speech-to-speech gateway
Later phase — v1 ships on ElevenLabs. The gateway is built when we move off it for performance and cost; the provider pattern is designed so it drops in as another provider, not a rewrite.
Our self-hosted voice-agent runtime, a peer of ElevenLabs. It accepts the Twilio media socket, runs the model's realtime protocol, owns the per-model audio handling and turn detection, and holds the agent configuration keyed by agent ID. It captures the transcript from the model stream and posts it back, and when the agent decides to transfer or end the call it calls the telephony service's tool endpoints.
On LiteLLM
The PoC reached the models through a single LiteLLM proxy, which was convenient for moving fast. Production will most likely drop it. It is not a full abstraction: it does not translate the audio semantics or the provider-specific realtime protocol details, so the per-model integration work lives in the gateway regardless of whether the proxy is there. And it is an extra network hop on the live audio path, which adds latency and is a potential bottleneck under load. The gateway will integrate each model's realtime API directly, which is where that work belongs.
Securing the media socket
The media socket is public (Twilio connects to it), so a URL token gates it. This is how ElevenLabs does it, and we mirror it.
ElevenLabs. We make an authenticated register call (our API key) naming the agent. ElevenLabs binds a conversation to that agent and returns a stream URL with a short-lived, per-call signed token embedded. The authenticated register is the trust anchor; the token is what Twilio presents on connect, carrying the agent binding and replay protection. ElevenLabs never verifies "this is Twilio", it verifies the token it issued.
Our gateway, same shape. The service makes an authenticated, service-to-service register call to the gateway with the agent ID and call metadata; the gateway mints a short-lived, single-use signed token and returns wss://gateway/media?t=<token>; the service returns that as the <Connect><Stream> URL; the gateway validates the token at the WebSocket handshake and rejects anything bad, expired, or reused, before any media flows.
One short-lived token in the URL carries authorization, agent selection, and replay protection, checked at the handshake.
The decisions that mattered
Each had a tempting alternative. Naming why we passed on it is the point.
Twilio owns telephony; we run no SIP or media server
Alternative: self-hosted SIP / media (Asterisk, FreeSWITCH, raw RTP).
Self-hosting would give total control of the media, but it is a whole reliability and on-call surface we would then own for a feature whose value is intake, not infrastructure. Twilio gives PSTN, media, recording, and HA out of the box, and its <Connect><Stream> primitive is exactly the hook every provider needs. We accept vendor dependence and per-minute cost for a far smaller footprint.
Media always leaves the telephony service; one provider pattern
Alternative: terminating the media in the telephony service.
Holding the audio would keep everything in one process, but it would make the service stateful, so a routine deploy drops calls in progress. Instead the service registers the call, hands Twilio a token-guarded stream URL, and steps out. It stays stateless in the live sense and deploys freely; the warm, stateful piece is the provider (for raw models, our gateway).
The service holds only an agent reference
Alternative: storing agent prompts, voices, and model config in the telephony service.
Agent specifics belong with the runtime that executes them, exactly as they do for ElevenLabs. The service stores a provider and an agent ID and nothing more, staying a thin control plane. The provider owns and secures the agent config.
Recording and transcript live in S3; the database points to them
Alternative: storing transcript text in a Postgres column.
Twilio writes the recording straight to the firm's S3 bucket, and we treat the transcript the same way: the provider posts it back, we write it to S3, and the database row holds the S3 location plus light metadata. Media never bloats Postgres, and recordings and transcripts are handled uniformly. Retrieval is a presigned-URL fetch rather than a database read.
LiteLLM was a PoC convenience, likely dropped in production
Alternative kept for the PoC: one proxy in front of every model.
It sped up the PoC, but it is not a full abstraction (the per-model protocol and audio work stays in the gateway) and it adds a hop on the live audio path that costs latency and can bottleneck. The gateway will talk to each model's realtime API directly.
A call on the wire
The same after-hours call as the demo, as the developer sees it: the inbound webhook, the authenticated register, the token-guarded media socket, and the callbacks. Twilio webhooks are verified with X-Twilio-Signature; provider callbacks carry a Bearer token.
Reply is TwiML: a spoken notice, then start recording, then connect the media stream to the token URL. Nothing after <Connect>, so when the stream closes the call ends.
TwiML200 response to /voice/incoming
body
<Response>
<Say>Connecting you to our virtual assistant.</Say>
<Start><Recording recordingStatusCallback="/webhooks/recording-status"/></Start>
<Connect><Stream url="wss://gateway.internal/media?t=eyJhbGciOi…"/></Connect>
</Response>
WSTwilio → wss://gateway.internal/media?t=…
gateway
validate token at handshake → bind to agent ag_8842 + call CA7f3…21a
run the model's realtime protocol, relay audio both ways
Caller and agent talk. The gateway captures the transcript from the model stream to post back at the end.
POST/voice/tools/escalate_to_human (Bearer)
from gateway
{ "call_sid":"CA7f3…21a", "reason":"caller wants a partner" }
we do
look up firm escalation numbers → redirect the live Twilio call
recording → S3, store recording row (s3 location, duration)
transcript → S3, store transcript row (s3 location, summary)
Recording and transcription
Recording is a Twilio responsibility end to end; audio never passes through the service for it. A <Start><Recording> at answer is non-blocking, so the call connects immediately, and it records the whole call including a human handoff. Twilio External S3 Storage writes the media straight to the firm's bucket; we store the S3 location in a recording row and hand the platform a short-lived presigned GET for playback, so the browser streams directly from S3.
Transcription works the same way. The provider transcribes the conversation and posts it back (Bearer webhook, correlated by call SID); we write the transcript to S3, exactly like a recording, and the transcript row holds the S3 location plus light metadata. Keeping the full text in S3 keeps the database lean and treats transcripts and recordings uniformly.
Source
How it is produced
Delivery
Storage
elevenlabs
ElevenLabs transcribes its own conversation and returns turns plus a summary
post-call webhook (Bearer)
S3 object; DB row points to it
gateway
The gateway captures transcript off the model stream
post-call webhook (Bearer)
S3 object; DB row points to it
How the agent ends or transfers a call
Only the telephony service holds the Twilio credentials and the call reference, so only it can transfer or hang up a call. Every provider therefore reaches the same Bearer-protected tool endpoints: the provider decides, the service acts. There are two channels per call: media (Twilio to provider) and control (provider to the service over authenticated HTTPS). The agent's call_sid rides in the register step, so the provider can name the right call when it calls back.
Media and control are separate: the provider holds the audio, the service holds the phone call.
Our speech-to-speech gateway
Managed platform (ElevenLabs)
Agent runs
On our self-hosted gateway
On the platform's servers
Media path
Twilio → gateway (token URL)
Twilio → ElevenLabs (token URL)
Control path
Outbound HTTPS callback to our endpoint
Outbound HTTPS webhook to our endpoint
Auth
Bearer token multi-token, rotatable
Bearer token multi-token, rotatable
Integration with the LawHive platform
BeeLine plugs into the LawHive platform (platform-v3): a TypeScript microservices monorepo with an Inngest event bus, per-service Postgres, Clerk auth, and an internal-ops app (admin-app). Two questions decide the integration: who owns which data, so nothing drifts, and how the two systems talk.
One writer per fact
The platform already owns the firm. BeeLine owns all telephony configuration. Neither fact is copied into the other system, so there is nothing to reconcile. BeeLine keys its firm row by the platform firm_id directly (no second id, no mapping table) and stores no firm name. The one shared value is firm_id, which the platform owns; a foreign key cannot drift the way a duplicated field can.
Config writes (numbers, hours, routing) and live reads for the call list, detail, recording, and transcript. platform-api is a thin proxy that stores nothing; nothing is materialized into the platform just to display it.
2 · Firm lifecycleasync · Inngest
identity-service→Inngest→BeeLine
firm.deleted / merged → BeeLine deactivates or repoints telephony. The one signal BeeLine must consume, so a removed firm doesn't leave live numbers routing.
3 · Call completionasync · Inngest
BeeLine→Inngest→downstream
Emits beeline/call.completed carrying pointers (call_sid, outcome, recording_s3, transcript_s3) for services that process the call: ingestion → RAG, matter / lead attach. They pull the full data from BeeLine or S3. Not the UI path.
Two owners, three seams. Reads are synchronous to the source of truth; the event bus notifies services that have downstream work.
Provisioning a number
Numbers are bought out of band in Twilio, then handed to Ops. Ops enters the E.164 in admin-app and assigns it to a firm; BeeLine validates the number exists in the Twilio account and isn't already routed, sets its voiceUrl to /voice/incoming, and reports pending → active (or an error) back to the UI. That status doubles as a drift guard: if a number's webhook is changed underneath BeeLine, it surfaces instead of silently failing a live call.
Where the UI lives
The management UI is a "Voice" tab on the firm's detail page in admin-app, not a standalone app. It reuses the platform's Clerk SSO, RBAC (a can_manage_voice_config grant), design system, and forms, and talks to BeeLine through the platform-api gateway. The standalone PoC console is superseded by it.
BeeLine makes no synchronous read from the platform on the call path, so a live call never depends on platform uptime.
dedicated BeeLine service credential (KMS-signed or BeeLine-scoped secret), not the platform-wide internal JWT
Inngest → BeeLine (firm events)
Inngest signing key
BeeLine → Inngest (call events)
Inngest event key
Twilio → BeeLine
X-Twilio-Signature
provider / agent → BeeLine tool endpoints
Bearer, multi-token for zero-downtime rotation
v1 scope
v1 ships on ElevenLabs only. With a managed provider BeeLine holds no media at all, so it is pure HTTP (no long-lived sockets to keep warm, nothing to drain on deploy) and the high-availability bar is easy to meet; transcription comes from the ElevenLabs post-call webhook. The self-hosted gateway is a later phase, built when moving off ElevenLabs for performance and cost. Because every provider uses the same register → token-URL → Bearer-callback seam, adding it is a new provider row, not a rewrite.
What comes next, what is deferred
Business tools beyond call control (matter lookup, booking, eligibility) are not telephony concerns, so they belong in a separate public tools service that owns the tool definitions and their execution, and serves both providers the same way: each calls it as an authenticated webhook. Same auth split as call-control tools.
Data model
Two owners, no shared copies. The platform owns firm identity; BeeLine is authoritative for everything telephony and keys its firm row by the platform firm_id. Media lives in S3 with rows pointing at it. The platform's firm is referenced, not stored by BeeLine.
Entity (BeeLine)
Key fields
Notes
firm
firm_id = platform firm id (PK), is_active
no name stored; fetched from the platform for display
Security. Every Twilio webhook verifies its X-Twilio-Signature. Provider callbacks (tools, post-call transcript) use Bearer tokens, several valid at once for zero-downtime rotation. The media socket is gated by the short-lived URL token. Calls into BeeLine from the platform use a dedicated BeeLine credential (not the platform-wide internal JWT). Presigned S3 URLs are short-lived. No customer data or live credentials live in this document.