Incident write-ups: timeline, impact, what went wrong, what we’d change. Bigger and more retrospective than an RCA — pairs well with an RCA when one led to the other.
| Date | Title | Author |
|---|---|---|
| 2026-08-24 |
Case-context saturation on a normal Monday
lawrence-engine ran at 100% CPU for three hours under ordinary Monday traffic. Friday's deploy had turned permanently-failing case-context work on email-heavy cases into multi-pass folds that two 1-vCPU tasks could not absorb. Found by a form upload that never appeared; no alert fired.
|
Adolfo |
| 2026-08-05 |
Document retrieval on large cases
One defect, two failures: retrieving a document cost as much as reading the whole case. Quietly, that hid documents from Lawrence; loudly, it exhausted file descriptors and crash-looped ingestion-management for two hours. Fixed by making the cost match the document.
Pairs with the document retrieval RCA, which covers the quiet phase.
|
Adolfo |