Stopped Guessing How Much Room Is Left

Killed the context-window estimate today. Not patched it again — deleted it. Every guess about how much room is left in a conversation is gone from the code now; the only number that matters is the one the provider hands back after a response. I’d gone back into this logic twice before to close a door a live-but-quiet host or a refused ping had opened. Today I stopped closing doors and pulled the estimating engine out entirely. The gate now reads real usage: sum every prompt-side counter the provider reports — a cached token lives under its own counter, not the input count, miss that and a long cached conversation looks smaller than it actually is — hold a request at 90% of the model’s real context window, and fire compaction before the send. No dependency propping up a guess for OpenAI-compatible hosts. No five separate estimate functions spread across five different provider clients. The provider’s own over-limit rejection is still the backstop, but a backstop is all it should ever have been.

Memory got the bigger swing, and the better story. Spent most of the day building the redesigned consolidator: Graph and Map stores replace the old fact/episode tables, and a background pass now walks unconsolidated conversation turns oldest-first and distills them into the new shape. First move was a migration to backfill the old data in before the switch. A few commits later, rewrote that migration into a thinner replay loop. By evening, deleted it outright — once the consolidator can walk every historical turn itself and mark it done, a script that only ever runs once is just a liability sitting in the repo. Built it, watched a simpler idea make it redundant, cut it same day. That’s the right call every time it happens.

One more piece landed on top: background research now writes into the same Graph and Map stores instead of a separate memory tool, and every research pass on the same topic now shares one stable identifier instead of forking a new one each time it fires. That was the gap that made a multi-day research thread read as a pile of disconnected notes instead of one investigation getting deeper. Fixed now — recall before you write, save after, same thread every time.

Ten more passes today converted procedural, module-level functions into classes across the backend — services, abilities, capabilities, the API layer, the LLM clients, cron, workers. A boot sequence that used to be thirteen loose helper functions is now one class. Nothing behavioral moved, every pass fully typed, zero new escape hatches. Not exciting work. It pays for itself the next time someone has to find where a function actually lives instead of grepping five files that all import each other’s helpers.

Smaller stuff rode along. Three capability classes had been hand-rolling the same YAML-manifest loader, and five file-system operations were each mapping the same permission error to the same response by hand — both folded into one shared implementation apiece. And the MCP SDK’s breaking 1.x changes finally caught up with the backend: the old server class doesn’t exist anymore, tool schemas got renamed, the streaming client hands back one more value than it used to. Fixed everywhere it broke.

Next real test isn’t today — it’s the next time a provider goes quiet mid-conversation, and whether “sized only if it answers” holds up outside a controlled day of rewrites the way the last two attempts at this exact fix didn’t.

• Deleted context-window size estimation entirely — the gate now reacts only to what the provider reports back, no guessing before a request goes out

• Rebuilt the memory consolidator around Graph and Map stores with a 30-minute background pass, then deleted the one-time migration it shipped with hours earlier once the consolidator could replay history itself

• Shipped the consolidator’s four tools (save, save narrative, delete, recall) and a recall service fusing keyword search with vector search

• Converted ten more waves of procedural functions into classes across services, abilities, the API layer, and the LLM clients — pure relocation, fully typed

• Folded two duplicated code paths (manifest loading, file-system error mapping) into shared implementations, and fixed the backend for the MCP SDK’s breaking 1.x API changes