April 13, 2026
Refactor: Document Handling & Memory Pipeline
Documents are now split into 512-1024 char overlapping artifacts stored in data_graph, replacing the heavy document_processing_service
Documents are now split into 512-1024 char overlapping artifacts stored in data_graph, replacing the heavy document_processing_service.py.
The data_graph service now supports KIND_DOCUMENT artifacts, which are hard-deleted without TTL or decay.
Document creation and searching are rewired to use artifacts in data_graph instead of the old document_chunks.
Artifacts are integrated into memory recall, surfacing the top-3 labeled document fragments.
A hard-delete cascade ensures all artifacts are removed when a document is deleted, and a new hard_delete_by_source_prefix() is available.
The old document_processing_service, document_classification_service, and document_queue are removed.
FTS5 delete command is now used for external content tables to prevent index corruption, and ordering bugs in deletions are fixed.
The system was cleaned up by removing the activity feed, DMN interaction log writes, and obsolete memory services.
-
Replaced heavy processing pipeline with data_graph artifacts for documents.
-
Artifacts are stored as kind=‘document’ in data_graph with hard delete policy.
-
Integrated document artifacts into memory recall, surfacing top-3 labeled fragments.
-
Switched external content table deletions to use FTS5 ‘delete’ command.
-
Removed obsolete activity feed and interaction log writes.
-
Replaced ExperienceAssimilationService with ToolSynthesisProcessor.