Memory First, Disk Later
The morning began with a question and an indignity. The question: what is the single biggest thing we could do to make LastDB fast? The indignity, uncovered by measurement: our entire logical dataset was smaller than a gigabyte, the machine had dozens free — and reading one record still cost 60% of scanning an entire column. We were paying disk prices for data that fit in a pocket. By midnight the fix was designed, built, merged, and running on the production database this company runs on. This is how the day went.
The production database in question is our own primary — the instance behind Brain and Kanban, the knowledge-base and task-board apps we build LastDB with. We dogfood everything; when reads are slow, our own tools are slow, and everyone here feels it before any customer would.
Measure before touching
The first hours produced no code, only numbers. Reading one record: about three seconds, and repetition did not help — it was structural work paid on every call, not a cold cache warming up. A single ordinary write triggered roughly two thousand cold storage-bucket loads, because the write path read broadly before it wrote narrowly. And the punchline already mentioned: all of this ceremony guarded less than a gigabyte of actual data.
Numbers like these do two jobs. They tell you the disease is structural, not incidental — no cache tuning rescues a read that re-derives the world each call. And they hand you the acceptance test in advance: if memory becomes the primary, a repeated read should cost a hash lookup, and the instrument should be able to prove it.
The design: resident-primary
Memory is the primary. The working set — the live graph of records — resides in RAM as typed objects, under a byte budget, and answers reads directly. Disk is write-behind durability: a background worker drains changes down; a miss loads from disk once, installs in memory, and the next read is a hit. This is not “disk with a cache in front.” The order of authority is inverted — memory first, disk later, cloud eventually.
The honest fine print: acknowledging a write before it reaches disk creates a crash window. We bounded it — the same posture as group-commit, measured in fractions of a second — and never evict anything from memory that has not been persisted. A budget with an eviction policy that refuses to drop unsaved work is the difference between a fast database and a fast way to lose data.
Seven changes, one working session
The build broke into small, separately shippable pieces — each one merged and green before the next landed on top:
- Stop paying a full durability barrier on every write batch — the deferred-flush machinery already owned durability; the barrier was a vestige.
- Put the resident graph under a byte budget with least-recently-used eviction of clean entries only, plus the background persist worker.
- Serve every read path from memory first, disk as the miss path — records are content-addressed, so a memory hit can never disagree with disk.
- Acknowledge writes after the in-memory apply; persist behind.
- Defer a synchronous bookkeeping write that measurement showed was half the remaining acknowledgment cost.
- Cap the deferred window, so a burst degrades to the slow-but-safe inline path instead of queueing unbounded work.
- Build the measurement harness that proves all of the above on real data.
Two of the seven were landed in parallel by our agent fleet — the autonomous engineering loop that builds LastDB alongside us — working from the same design notes and measurements. The pieces composed without a meeting: the fleet’s write-path change consumed the budget-and-worker foundation from earlier in the day as if one hand had written both.
The balloon
Within the first hour of memory-first writes running in production, memory usage jumped from 1.3 to 13.6 gigabytes in about sixty seconds, and the watchdog we keep pointed at the database did exactly what it exists to do: killed the process and restarted it. Ten seconds of downtime, no data corruption — the bounded-window design held — but a loud lesson: a fast acknowledgment without backpressure is a loan, and bursts collect.
The first fix capped the deferred queue by task count. Measurement promptly demoted it: the balloon reproduced on a binary without the new write path at all, so the count cap was aimed at the wrong unit. The durable fix — one accounted byte budget across everything the deferral holds — landed the same night. We kept the incident record, the wrong first diagnosis and all. The record that survives revision is the one worth keeping.
Measure like you mean it
Every claim in this post was measured the same way: boot the candidate against a copy-on-write clone of the real production database — never a synthetic fixture, never the live node — and run the same instrument over both the old and new builds. Wall clocks lie in friendly ways, so the instrument also asserts counters: a warm read must perform zero cold storage loads, not merely feel quick. Phase timing inside the write path is what found the bookkeeping write hiding in plain sight, and it is also what keeps us honest about what remains: the storage work in a write acknowledgment is now fifteen to twenty milliseconds, and most of what the caller still waits on is request plumbing around it — named, measured, and next.
Point reads: 2,970 to 48 milliseconds. Column scans: 10,300 to 2,100. The write pipeline: seconds to under twenty milliseconds. Same data, same machine, one working session apart.
The shape of the day
Breakfast: a question. Morning: measurements that made the answer obvious. Afternoon: seven scoped changes, each merged behind its own tests. Evening: a green-gated cutover of the production database, an incident caught by its own guardrail, and a fix-forward before the night ended. No war room, no freeze, no migration — the same database, remembering that the fastest place to keep data you already hold is where you are already holding it.
Built with Brain and Kanban — open-source apps on LastDB — inside our autonomous build loop.