Compaction goes compact-first, thinking gets un-stuck

Yesterday’s ability-framework split cleared the ground for the harder problem underneath it: what happens when a conversation gets too big for the model’s window. Today was about making that moment invisible instead of a crash.

The old design waited for trouble — send the request, catch the overflow, trim the oldest messages, retry. It was reactive by construction: a user could hit a turn that rendered a partial view of their own history because the trim happened after the fact. The rewrite flips the order: every iteration measures the full request before it goes anywhere; if it would blow the window, compaction fires first, the watermark advances, and the request is rebuilt collapsed — never sent half-formed. Providers.send() gained a plain force flag: force=False returns an “over cap” signal without touching the API; force=True bypasses the check so a request that genuinely can’t shrink further fails loudly at the provider instead of looping forever. The old trim path and its overflow-exception machinery are gone; convergence is now two iterations, not a chase.

The more interesting move is how compaction runs: through the same tool-dispatch path everything else in Chalie uses, as two internal abilities the model never sees or calls directly — one for chat history, one for the reasoning trail. Routing it through the normal dispatcher instead of a bespoke method fixed three gaps at once: compaction now shows in the visible reasoning trail, it shows in the Brain panel that lets you inspect what the assistant actually did, and the response-routing bug from the old bolt-on path disappeared with it. The compacted summary is now its own transcript row, and that row’s ID is the bookmark for “everything before this point has been folded in” — no separate tracking table, no drift between stored and marked-read.

A sharp edge worth being honest about: moving the bookmark reset every channel’s marker to zero on existing conversations, and one live thread rendered its entire history — 1.76 million tokens — into a single request, blowing straight through the window it was meant to protect. The fix was a hard cap read from one constant everywhere it matters, so render size and trigger threshold can never disagree again — the kind of bug that only shows up in production-shaped data, why “does this hold up on a real conversation” stays part of how I judge a change before calling it done.

Also buried in the pass: a genuinely embarrassing dead wire where the setting that escalates the model into high-effort thinking was written to a private variable nothing downstream read, while the dispatch code checked a public one instead — silently inert on every turn meant to trigger it. One-line fix, now pinned down by a regression test that fails on the old code and passes on the new.

Smaller but real: tool-discovery search got a quality floor so weak semantic matches stop getting injected just because nothing better ranked higher, and the composer gained a thinking-level override to force deeper reasoning on demand plus a running indicator of context-window usage — the difference between a black box and a system you can reason about while using it.

None of this changes what Chalie can do today. It changes whether long conversations stay coherent instead of quietly forgetting themselves — the whole point of building something meant to be lived with, not just chatted with once.

• Compaction rewritten compact-first: every turn measures the full request and compacts before sending, never after overflowing
• Compaction now runs through the same dispatch path as every other action, so it’s visible in the reasoning trail instead of hidden machinery
• A production overflow (1.76M tokens in one request) got traced to a bookmark reset and closed with a single shared cap, not a patch
• High-effort thinking had been silently inert due to a variable-naming mismatch between the setting and the code that reads it — now fixed and covered by a regression test
• Composer gained a thinking-level override and a live context-usage indicator, so what the model is doing is visible, not assumed