April 22, 2026
Perf boost for batch embeddings
generate_embeddings_batch now checks MemoryStore per text before encoding, deduplicates repeated strings, and encodes only unique misses in chunks of 32
generate_embeddings_batch now checks MemoryStore per text before encoding, deduplicates repeated strings, and encodes only unique misses in chunks of 32. Cache hits reuse the stored vector; fresh computes are written back. Input order is preserved. This helps in consolidation flows where the same key string recurs across the batch.
-
Cache lookup in batch path with de-dup
-
Checks MemoryStore per text before encoding
-
Deduplicates repeated strings
-
Encodes only unique misses in chunks of 32