Chalie Goes Outside

Ambient Tool Intelligence

Until today, Chalie’s cognitive drift and curiosity threads were purely internal — they’d surface interesting thoughts and explore threads, but only using knowledge already inside the system. That changes now.

We shipped the AmbientToolAction, a new autonomous action that bridges Chalie’s internal cognition to external on-demand tools. When the drift engine surfaces a thought or the curiosity engine picks up a thread, it can now evaluate whether an external tool (Reddit, News, SearXNG, etc.) might enrich the exploration — and invoke it if so. The model is opt-out: every on-demand tool in the registry is ambient-eligible by default. Tools that don’t make sense for background invocation can opt out via a single manifest flag.

This is deliberately constrained. Ambient tool invocations go through the same confidence gating and cooldown logic as other autonomous actions — Chalie won’t spam external services, and the results feed back into memory the same way ACT loop results do. The intelligence grows from the outside in, but always through the same memory pipeline.

A related fix enforced a key tool-agnosticism principle: ambient tool manifests no longer accept default_params. Tools should never configure how the infrastructure invokes them — parameter decisions belong to the invoking action, not the tool. The fix removed this escape hatch entirely.

Watched Folders and Vision OCR

Documents have been a persistent pain point: the old pipeline processed files serially, retried failed jobs in a tight loop, and had thread-safety issues around embedding generation. Today’s work replaced the entire local ingestion path.

The new FolderWatcherService monitors configured directories for new and modified files, handing them off to a proper concurrent queue backed by three worker threads. Failed files are not re-queued unless they’re actually modified on disk — ending the infinite retry spiral that had been processing 2 of 249 test documents before giving up. Embeddings are now generated behind a thread-safe lock, eliminating a class of race conditions that had been silently corrupting vectors.

Alongside this, we built a full Vision LLM OCR pipeline (OcrService) for extracting text from scanned documents and image-heavy PDFs. It supports Anthropic, OpenAI, Gemini, and Ollama as providers, with graceful fallback when vision isn’t available. The brain admin panel now includes a Watched Folders UI so you can configure and monitor which directories Chalie is watching.

Several bugs were also caught and fixed in the new code before they could compound: an incorrect class name (ProviderDBService → ProviderDbService) was causing OCR provider resolution to fail silently; the document_chunks INSERT was incorrectly providing an id column that’s AUTOINCREMENT; and find_duplicates was querying summary_embedding IS NOT NULL on the documents table even though that column lives in the documents_vec virtual table.

Document Classification and Data Model Cleanup

New documents now get automatically classified by project and category using an LLM pass during ingestion. A meta_locked flag prevents re-classification once a user has manually edited the metadata — the system learns to leave human judgment alone.

We also unified two overlapping data models: “Moments” (pinned chat messages) are now stored as a document type rather than in their own table. This simplifies the backend considerably, brings moments into the hybrid search index, and removes a category of special-case handling that had been accumulating across services.

A handful of time_utils improvements landed as well, rounding out the datetime standard compliance across the folder watcher and episodic retrieval paths.