Lists Stop Guessing, and I Learn to Stop Guessing Too

Today was about two different flavors of the same lesson: when Chalie guesses instead of asking, it fails quietly, and quiet failures are the worst kind because nobody notices until the numbers already dropped.

The first one was the list ability. Our own testing flagged that list handling had regressed hard — from a 0.78 pass rate down to 0.29. The cause was almost embarrassing: the old schema told the model it could “omit name” for add, remove, check, or view actions, and when it did, the system silently fell back to whichever list was most recently touched. That’s a textbook polite hallucination — the model sounds confident, picks a list, and it’s just not the one you meant. No error, no clarification, just the wrong basket getting the wrong item. I rebuilt ListAbility from scratch as a strict CRUD interface: every list now gets an 8-character hex ID the moment it’s created, and every action past creation — view, add, check, remove, clear, rename, delete — requires that ID explicitly. The model has to call list_all first and actually see what exists before it touches anything. No more magic “most recent list” fallback, no more silent successes on a check or remove that matched nothing. If you ask to remove an item that isn’t there, you now get told that, instead of Chalie nodding along. I also dropped the list-history table entirely — it existed to power the guessing, so once the guessing was gone, so was the table. Rewrote the rich-media wrapper so list cards use the same wire format as every other tool, and locked the contract with a new test.

The second thread was messier and more honest about how this kind of work goes. A different case — a simple “look up the Apple earnings report” query — had also regressed, from 0.96 down to 0.69, because an earlier expansion of the subagent’s description made it sound like the obvious choice for any external lookup, including ones that should’ve gone straight to a dedicated search or news tool. My first instinct was a broad fix: add an explicit “discover tools before delegating” rule to the system prompt, steer the subagent’s own docs away from single lookups, and sharpen the tool-finder’s pitch so it could compete for attention. I shipped it, then reverted it the same day. It solved the routing problem but at the cost of bloating the prompt with a new operating principle for what was really a narrow, local ambiguity. The fix that stuck was much smaller: tighten the subagent’s summary with one leading clause — it’s for multi-step actions or high-volume processing, full stop, single-tool lookups go straight to the tool. One sentence instead of a new system-wide rule. Small change, but it’s the difference between patching a symptom everywhere and fixing the actual ambiguity where it lives.

Both fixes point at the same instinct I want baked into everything Chalie does: never let the model paper over a decision it isn’t sure about. If it doesn’t know which list you mean, make it look first. If it doesn’t know which tool is right, don’t give it a reason to hedge toward the biggest hammer in the box. Determinism where determinism is possible, ambiguity surfaced instead of silently resolved in the model’s favor. That’s what “it just works” actually requires — not a smarter guess, just fewer places where guessing was ever an option.

Next up is watching how these two changes hold up over the following days — list handling and single-tool routing were both blind spots that only showed up because they were being watched for regressions in the first place, which is exactly the discipline I want to keep expanding.

• Rebuilt ListAbility as strict CRUD, addressed end-to-end by 8-char hex ID, no name-based fallback
• Removed the “most recently used list” guess that was silently editing the wrong list
• Dropped the list-history table now that nothing depends on it for recovery
• Tightened the subagent’s own description instead of adding a new system-wide routing rule
• Locked both fixes behind unit tests so the next regression shows up before it ships