Published study · July 2026
Agent Memory Ablation
Vector-free graph retrieval refused 100% of adversarial distractors. Hybrid retrieval, on the same corpus and the same harness, refused 10%. Two of four pre-registered hypotheses were refuted and published anyway.
Why the finding is trustworthy
- A deterministic 1,583-fact corpus at seed 42 with fully known ground truth, checksummed with SHA-256.
- 200 queries split deliberately: 150 known-item, 30 multi-hop across two or three joins, and 20 adversarial distractors where no correct answer exists at all.
- The query set was built before any arm was run.
- All three arms (stateless RAG, knowledge graph, hybrid) sit behind one shared typed interface, so the comparison is fair rather than favorable.
- Reproduces bit for bit from a single command.
Results
- Known-item Recall@5, graph
- 98.7%
- Known-item Recall@5, hybrid
- 98.0%
- Known-item Recall@5, stateless RAG
- 68.7%
- Refusal on adversarial distractors, graph
- 100.0%
- Refusal on adversarial distractors, hybrid and RAG
- 10.0%
- Pre-registered hypotheses refuted
- 2 of 4
What came back against me
- Two of the four pre-registered hypotheses were refuted. Both are in the results table with the rest.
- The arm I expected to win did not. Adding a vector channel to the graph made distractor refusal ten times worse, which is the opposite of the intuition that more retrieval channels means better recall.