Research

Paper

AI LLM March 16, 2026

SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval

Authors

Jesper Derehag, Carlos Calva, Timmy Ghiurau

Abstract

Recent conversational memory systems invest heavily in LLM-based structuring at ingestion time and learned retrieval policies at query time. We show that neither is necessary. SmartSearch retrieves from raw, unstructured conversation history using a fully deterministic pipeline: NER-weighted substring matching for recall, rule-based entity discovery for multi-hop expansion, and a CrossEncoder+ColBERT rank fusion stage -- the only learned component -- running on CPU in ~650ms. Oracle analysis on two benchmarks identifies a compilation bottleneck: retrieval recall reaches 98.6%, but without intelligent ranking only 22.5% of gold evidence survives truncation to the token budget. With score-adaptive truncation and no per-dataset tuning, SmartSearch achieves 93.5% on LoCoMo and 88.4% on LongMemEval-S, exceeding all known memory systems under the same evaluation protocol on both benchmarks while using 8.5x fewer tokens than full-context baselines.

Metadata

arXiv ID: 2603.15599
Provider: ARXIV
Primary Category: cs.LG
Published: 2026-03-16
Fetched: 2026-03-17 06:02

Related papers

Raw Data (Debug)
{
  "raw_xml": "<entry>\n    <id>http://arxiv.org/abs/2603.15599v1</id>\n    <title>SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval</title>\n    <updated>2026-03-16T17:53:21Z</updated>\n    <link href='https://arxiv.org/abs/2603.15599v1' rel='alternate' type='text/html'/>\n    <link href='https://arxiv.org/pdf/2603.15599v1' rel='related' title='pdf' type='application/pdf'/>\n    <summary>Recent conversational memory systems invest heavily in LLM-based structuring at ingestion time and learned retrieval policies at query time. We show that neither is necessary. SmartSearch retrieves from raw, unstructured conversation history using a fully deterministic pipeline: NER-weighted substring matching for recall, rule-based entity discovery for multi-hop expansion, and a CrossEncoder+ColBERT rank fusion stage -- the only learned component -- running on CPU in ~650ms. Oracle analysis on two benchmarks identifies a compilation bottleneck: retrieval recall reaches 98.6%, but without intelligent ranking only 22.5% of gold evidence survives truncation to the token budget. With score-adaptive truncation and no per-dataset tuning, SmartSearch achieves 93.5% on LoCoMo and 88.4% on LongMemEval-S, exceeding all known memory systems under the same evaluation protocol on both benchmarks while using 8.5x fewer tokens than full-context baselines.</summary>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.LG'/>\n    <published>2026-03-16T17:53:21Z</published>\n    <arxiv:primary_category term='cs.LG'/>\n    <author>\n      <name>Jesper Derehag</name>\n    </author>\n    <author>\n      <name>Carlos Calva</name>\n    </author>\n    <author>\n      <name>Timmy Ghiurau</name>\n    </author>\n  </entry>"
}