Counting cannot bind

Theory update · 2026-06-28 · from compression to comprehension · experiment comprehension, slice 1

Six straight rounds on prediction said the same thing: the next symbol is a local game counting already wins, and a more abstract code does not predict it better. So the program changed the question. The goal is a brain-inspired cognitive architecture we can talk to, and the smallest version of talking to it is not compression but comprehension: say two facts, ask a question, get the answer. "Here is an apple. It is green. Here is a cup. It is blue. What color is the apple?" The convergent finding from a four-stream fan-out is that this is not a prediction problem at all. It is a variable-binding, one-shot relational-memory problem, and our two engines, the count predictor and the apical abstractor, are both the neocortex: slow, dense, structure pulled from many examples. The step the task turns on, bind this entity to this property on a single encounter and read it back from a cue, is the defining job of a different organ the brain runs in parallel and we did not have: the hippocampus, fast, sparse, content-addressable. This round builds the smallest version of that organ, a gradient-free binder, and tests it on the apple/cup task with one control that makes the result mean something. The property is randomized every episode, so corpus-wide every entity is equally often every color, which makes any counting or co-occurrence model provably sit at chance: only a model that reads this episode's binding can answer. And the result is the first positive of the new architecture. Counting is pinned at chance (0.246 against a chance of 0.250). The gradient-free binder answers perfectly (1.000). Scramble its store and it collapses back to chance (0.243), so the store does the work, not a leak. The smallest thing we can talk to, working, in regime: online, gradient-free, bounded.

The question

The program has spent six rounds learning, carefully, what it is not for. Next-character prediction is a local game. A plain n-gram beats us on it and the gap widens with data. The rank-reuse tension showed a count head on the abstract code pays a stable half-bit tax, and abstraction cannot cheapen prediction showed the feedback arrow from the abstraction engine to the prediction engine is null for the next character: where the tag carries information it is redundant with the n-gram, and where it brings new context it has no next-character signal. The honest reading of that last round was not that the work failed but that we were holding the wrong ruler.

So a four-stream research fan-out, with the author's own notes, asked what the right ruler is. What is the bottleneck, if not prediction? Four independent streams returned the same answer, and so did the old design sketch sitting in the repository. The goal is a brain-inspired cognitive architecture we can talk to, and the smallest honest version of "talking to it" is comprehension, not compression. Say a fact, ask a question, get the answer back. That is a thing you can hold in your hand and check, and it is the floor of every conversation.

The convergent finding is that this floor is a different kind of problem. "What color is the apple" is a variable-binding and one-shot relational-memory problem, not a prediction problem. You meet the apple once, you bind it to green once, and later you read green back when asked. Our two engines, the count voting bank that predicts and the sparse apical stack that abstracts, are in brain terms the same organ: the neocortex, the slow system that extracts structure over many examples. The load-bearing step here, bind this entity to this attribute on a single encounter, keep it separable from the cup, and complete it from a cue, is the defining job of the organ the brain runs alongside the neocortex and the one we never built: the hippocampus, fast, sparse, pattern-separated, content-addressable. This is exactly the honest result the program already had, the architecture is the bottleneck, now with the missing piece named.

And the encouraging half: there is a concrete mechanism that fits all four of the program's rules (online, gradient-free, bounded, built on the same sparse codes we already emit). A statement binds a role to a filler, a question unbinds the role, and the answer is cleanup to the nearest stored item. This is vector-symbolic binding, the engineering form of the hippocampal factorization that the Tolman-Eichenbaum Machine spells out in the biology. The missing organ is not a vague aspiration. It is buildable, with the parts we have, now.

The bet, and the instrument

The bet, stated to be falsifiable. A gradient-free, sparse, content-addressable binder, the smallest hippocampus, answers the apple/cup task, and the counting engine we already have provably cannot.

That second clause is the whole design, and it rests on one control. The instrument is a generator that emits episodes in real letters: "here is the apple stop it is green stop ... what color is the apple ask," with stop and ask as in-alphabet terminators so the character and word structure is real. Stage zero, one, and two: four entities, four colors, so chance is one in four. Twenty thousand episodes, scored on ten thousand held-out questions, each about a binding the model is seeing for the first time, in an episode it has never seen.

The control is per-episode randomization. Which color attaches to which entity is resampled every single episode, so over the whole corpus the apple is green exactly as often as it is red, blue, or black. The marginal probability of a color given an entity is flat by construction. So any model that conditions only on corpus statistics, no matter how clever its counting, is provably at chance. The only way to beat chance is to read the binding stated in this episode. That single line is what turns a comprehension demo into an architecture test: a pass means binding, not frequency-guessing.

Every model reads the same parsed stream of facts. They differ only in how they store and retrieve. Five of them:

• Uniform. Answer a random color. The floor.
• Most-frequent-given-entity. The strongest a co-occurrence reader can be: the color most often seen with the entity, counted over all of training. The control is engineered to flatten exactly this.
• A dictionary binder. An exact, episode-local Python dictionary from entity to color. Not a model of anything, a validator: it confirms the task is solvable and the scorer is right.
• The corrupt-store control. The real binder, but with the store loaded with a scrambled set of bindings before the question. If the answer survives a scrambled store, something other than the store is leaking the answer, and the whole result is void.
• The VSA binder. The gradient-free one. Each word gets a fixed random high-dimensional code (the minimal stand-in for a cortical code). A statement binds the entity-and-role to the color and superposes it into one episode trace; a question unbinds the entity-and-role and cleans up to the nearest color. No gradients, no training of the codes, no learned weights at all. Pure binding.

What happened

The picture is the whole finding.

The numbers, held-out QA accuracy, chance 0.250:

Read it in order. Counting is pinned at chance. Most-frequent-given-entity, the best a counting reader can do, scores 0.246 against a chance of 0.250, dead on the line. The randomization control did its job: there is no co-occurrence statistic to lean on, because the corpus made every entity equally every color. This is not a weak baseline losing; it is a provably unwinnable game for counting, and the number confirms the proof held in the data.

The instrument is valid. The exact dictionary scores 1.000, so the task is solvable and the scorer reads the right slot. And the store is what does the work: scramble it and the binder collapses to 0.243, right back to chance, so nothing is leaking the answer through a side channel.

And the gradient-free binder answers perfectly. 1.000. It clears the binding bar a roughly three-year-old child sets, around 0.75 to 0.90, with room to spare, and it does it the way the brain's fast system does: one shot, sparse, content-addressable, no gradient anywhere. The count engine the cortex already has provably cannot do this; the hippocampal binder it lacked does it cleanly.

One more probe, because a binder that holds exactly two facts is a toy. Capacity. Push more bindings into a single episode and watch the dimension-versus-load tradeoff that vector-symbolic memory is known for. At the working width of two thousand and forty-eight dimensions, eight entities still scores 1.000, and eleven entities still scores 1.000. Shrink the codes hard, to two hundred and fifty-six dimensions, and eleven entities holds at 0.999. Only at a tiny sixty-four dimensions does it finally bend, to 0.884, the graceful capacity falloff exactly where the theory says it should be, far below the width the binder actually runs at. So it holds a roomful of facts, not just two.

What this means

This is the first positive of the neocortex-plus-hippocampus architecture. For six rounds the program measured what its two engines could not do, and named the missing organ. This round built the smallest version of that organ and watched it do the job the existing engines provably cannot. The gradient-free, sparse, content-addressable relational binder comprehends; the count engine, on the very same stream, sits at chance. The neocortex has it not. The hippocampal binder does what counting can't.

It is worth being precise about why this is not just a hash map with good PR. The control is the point. A hash map would also pass, and that is fine, because the question this slice answers is not "is binding hard" but "does the cortex's own gradient-free substrate bind, in regime, under a control that makes counting provably fail." It does. The same four operations, bind, bundle, unbind, clean up, that the brain's fast system is theorized to run, executed on the sparse codes we already emit, with no gradient and a bounded memory, answer the smallest question you can ask of something you want to talk to.

And it reframes the rest of the program as load-bearing rather than stalled. The data does not flow around the neocortex; it flows through it. The hippocampus does not bind raw letters, it binds the cortex's codes, which means the abstraction engine's representation quality, the transfer CCGP that climbed above the backprop ceiling with no gradient, becomes the dial on whether bindings generalize to entities never seen, the way the Tolman-Eichenbaum factorization rides new content on a shared structure. The abstraction result was never vanity. It is the next phase's generalization knob.

The program pivoted from compression to comprehension, and landed its first positive. The apple/cup task, say two facts and ask a question, is a variable-binding and one-shot relational-memory problem, the job of the hippocampus the program's two neocortical engines lacked. Under a control that randomizes the property every episode (so counting is provably at chance), the strongest co-occurrence reader scores 0.246 against a chance of 0.250, while a gradient-free, sparse, content-addressable VSA binder over random codes scores 1.000, clearing the three-year-old binding bar; corrupting its store collapses it to 0.243, so the store does the work. Capacity is graceful: eleven facts at full width still 1.000, bending only at a tiny sixty-four dimensions to 0.884. The first working piece of a cognitive architecture we can talk to, online, gradient-free, and bounded.

The honest caveats

• The fillers are random codes, not learned cortical codes. This tests the binding mechanism, not yet generalization to novel entities. That is the next phase, and the genuinely open bet: acquiring the relation and role structure online, without a gradient, where the Tolman-Eichenbaum Machine uses backprop. Phase one, this round, does not need it and is low-risk. Phase two is where the abstraction engine's code quality earns its keep, and where the gradient-free constraint bites hardest.
• Stage zero, one, and two, one property class. Color only. Typed binding (color versus size, so "what color" must not return the size), relations ("which is bigger"), negation ("is the box red"), and novel-entity fast-mapping are all untested, waiting up the curriculum ladder. Each is a stage that even strong neural models find hard, and the last is impossible by counting.
• It is a binder, and the slice proves an instrument. The dictionary binder passing is the validation that the task and scorer are sound; the VSA result is the architecture claim. Neither is fluent dialogue. This is the smallest thing we can talk to, with the emphasis on smallest.
• One seed, the headline numbers stated as run. The control, the collapse, and the capacity falloff are the structural reads, and they are exactly where theory predicts.

Lineage

Grew from abstraction cannot cheapen prediction and the five prediction-metric negatives before it, which together established that the next symbol is a local game counting already wins, so prediction is not the program's axis. The honest "we were holding the wrong ruler" at the end of that round is the door this one walks through. It reuses the program's standing parts: the sparse codes the substrate already emits, and the count organ as a ready cleanup memory.

Led to the pivot now recorded in the architecture: two neocortical engines (prediction and abstraction) plus a hippocampal relational binder, with a statement/question interface, a cognitive architecture you can hold a small and growing conversation with. The named next step is learned cortical codes in place of the random ones, the move that turns one-shot binding into one-shot generalization.

Thread: comprehension, and online learning without a global gradient. A first positive that names the missing organ and shows it works in regime: the gradient-free binder does the relational job the count engine provably cannot, and the control is what makes the claim mean binding rather than counting.