Building an agent memory that makes analogies

> The ceiling of similarity

Most memory systems for AI agents do one thing: embed the query, embed the store, and return the nearest neighbors. It works, and my system uses it too. But nearest-neighbor recall has a ceiling. It is very good at finding text that looks like your text, and blind to the moment that matters most in reasoning: recognizing that a new situation is like an old one even when the two look nothing alike on the surface.

That gap is where I started. If an agent is going to reason about how things relate, most similar is not enough. Sometimes the memory you want is the one that rhymes structurally with the present while wearing completely different clothes.

> Where the idea came from

The framing comes almost entirely from Douglas Hofstadter. In his 2001 essay Analogy as the Core of Cognition and the lecture of the same name, he argues that analogy is not a rare, occasional trick the mind performs. It is the basic machinery of thought. Every time we recognize that this is a kind of that, every time a past experience informs a present one, an analogy is being made. Categories themselves, in his account, are not fixed boxes we sort things into; they are built up continuously from noticing that several different things play the same role.

I read that and could not stop thinking about it as a design constraint rather than a philosophy. If analogy is the core of cognition, then a memory that only does surface similarity is missing the core. So I tried to build the missing part.

> The analogy slot

The result is a mechanism I call the analogy slot. On every turn, alongside the normal keyword and vector recall, the system is allowed to surface exactly one memory for a different reason: because it is structurally similar to the current situation while being superficially different.

Making structurally similar but surface different into something mechanical takes two different views of the same memory. For structure, I encode facts as Holographic Reduced Representations, phase vectors that bind each entity to its role, subject and relation and object, in a way where similarity survives the binding. That binding is what lets a fact like a company owns a subsidiary land close to a friend owns a car: both reduce to the same Entity OWNS Entity shape even though not a single word overlaps. For surface, I use an ordinary sentence embedding, the same one behind normal recall. A candidate qualifies only when the two disagree in a specific direction: high structural similarity under the holographic encoding, low surface similarity under the embedding. That disagreement is the signal. It is the difference between this reminds me of something and this is the same topic, and it is the shape of the human moment Hofstadter keeps pointing at.

One detail took a while to get right. Holographic superposition compresses pairwise similarities into a narrow band, so a fixed threshold does not work. The gate has to be relative. A candidate has to stand a couple of standard deviations above the mean similarity of the entire store before it is even checked against the surface ceiling.

Put together, the whole mechanism is two cheap operations: encode every memory twice, then look for the one place the two encodings disagree. Here is what that looks like on a real turn.

> Categories that make themselves

The second idea from the essay, that categories are made rather than given, shaped two more mechanisms. A weekly pass clusters facts that keep co-occurring and compresses each cluster into a single summary, without ever deleting the originals. Separately, graph nodes that play structurally identical roles, the same relations to the same kinds of neighbors, get proposed as instances of an emergent category that the system did not start with and that a language model names after the fact.

Neither of these happens silently. Chunks keep their members, and category proposals are tagged and reversible in a single call. The goal was categories that emerge from use while staying falsifiable, which is closer to how Hofstadter describes concepts actually forming than a fixed ontology decided in advance.

> The rest of the stack

Around those ideas sits a more conventional, and I hope well-engineered, memory stack. Structured facts, semantic vectors, and a knowledge graph all live in one SQLite file, maintained incrementally as a side effect of normal writes. Names are canonicalized at write time, so the same person or company collapses to one node no matter which source mentioned them. Forgetting is driven by use rather than age: a fact that gets surfaced and then engaged with is reinforced, a fact that is surfaced repeatedly and ignored quietly ranks lower, and nothing is ever pruned just for being old.

> What it does not claim

It is worth being honest about the ceiling. This is not a reimplementation of Hofstadter's Copycat, and it is not doing structure mapping over rich representations. The analogies it finds are bounded by what a bag-of-entities encoding can carry, which means it reasons over which entities and relations are present, not over deeper causal or structural patterns. It is two cheap, literal mechanisms drawn from a couple of his central claims, wired into an otherwise ordinary retrieval system. I think that is the honest description, and I also think the mechanisms earn their place.

> Try it

The whole system is open source, built as a memory provider plugin for NousResearch's hermes-agent. The code, the architecture notes, and a longer design write-up are on GitHub.

> Sources

The thinking here is downstream of two things, both worth your time:

• ▸Douglas Hofstadter, Analogy as the Core of Cognition (2001), in The Analogical Mind: Perspectives from Cognitive Science, MIT Press.
• ▸Douglas Hofstadter, the lecture of the same name.