What changed: the abstraction wall is credit assignment

Theory update · 2026-06-27 · the first positive on the abstraction wall · experiment DC

Six convergent negatives, from six different gradient-free mechanisms, all said the same thing: these rules carve structure and compress dimensionality but never build an abstract space. They pointed at one suspect, the gradient. This round tests it the only way that isolates it: one fixed architecture, same data, same online single-pass regime, same probe, swap only the update rule. The gradient builds the abstract space. The no-gradient rule does not. The wall was credit assignment, not topology.

What this round was

For the whole program, abstraction has been the wall. A run of experiments (an Oja embedding (CG), a predictive-coding stack (CH), a shared-representation hierarchy (CW), a pure-biology HTM voting cortex (CZ), temporal pooling (DA), and top-down Hebbian feedback (DB)) each took a different gradient-free, online, local mechanism and asked it to build an abstract code: low-dimensional, factorized, generalizing to cases it never saw, the geometry the CCGP probe rewards. Every one of them carved boundaries, compressed dimensionality, learned sequences, even settled and disambiguated a hierarchy. Not one made the space abstract.

Six negatives is a diagnosis, not just a list. They all pointed at the same suspect: the missing ingredient is specifically the gradient. The last of them, the top-down feedback, was the sharpest clue. It was a non-local credit signal, the escape the diagnosis had named, and it still failed. So the answer was not non-locality in general. This round tests the suspect directly.

The experiment

The only way to isolate the update rule is to hold everything else exactly fixed. So we built one small network (a context window of the last five characters, through the program's existing random sparse encoder, through one hidden layer, to a prediction of the next character) and trained it two ways with everything else identical. Same text slice, same seed, same online regime: one character at a time, in order, single pass, no epochs, no shuffling. True online, respecting the no-batching rule. The only difference between the two arms is how the hidden layer learns.

GRAD learns by online backprop. It is the global gradient oracle: online and single-pass, so it respects the no-batching rule, but global and non-local, so it is a measuring stick, not a shippable learner. HEBB is the identical network whose hidden layer learns by online Oja and the program's bounded local Hebbian readout, no gradient anywhere. We probed the hidden representation with the same three tools imported verbatim from the prior negatives, so the comparison is exact.

What happened

Holding architecture, input, online regime, and probe fixed, the gradient builds the abstract space and the no-gradient rule does not.

Read the two middle and right columns together; they are the whole result. GRAD's bits-per-char falls hard, which proves the gradient is wired correctly. Its abstraction score rises over training. Its dimensionality settles at a moderate 36. HEBB does the exact opposite on the part that matters: it compresses dimensionality harder than the gradient does, all the way to 14, the most aggressive collapse in the round, and its abstraction score falls below the raw-input baseline. That is the six-negatives signature, reproduced in one controlled cell: compress fiercely, lose abstraction. Consistent across three seeds; the gradient wins every time.

The verdict

Swap only the update rule, and the gradient builds a more abstract space while the no-gradient Oja arm compresses dimensionality hard but loses abstraction: the exact signature of the six negatives, now from the same architecture on the same data. The abstraction wall was credit assignment, not topology.

This is the first positive of the line. For months the program collected closed doors on abstraction and could only say "not this mechanism, not that one." DC says what the missing thing is. Better wiring alone does not close the gap; the update rule does. The six negatives were not six unlucky architectures. They were six demonstrations of one fact, and here it is isolated.

The honest caveats

We keep them in full, because the claim is strong and the evidence is one careful experiment.

• GRAD is the oracle, not the answer. It is global backprop: online and single-pass, but not local and not biological. It is a measuring stick that proves a gradient can build the abstract space, not a learner the program would ship. The program's four laws still forbid it.
• One architecture, one probe. A context-window network and one abstraction score, not the full stack. It shows that within a fixed topology the gradient beats no-gradient for abstraction. It does not prove topology is irrelevant; both can matter. What it kills is the claim that better wiring alone closes the gap.
• The absolute numbers are modest. Abstraction sits around 0.50 and barely rises during training on one label; the story is "the gradient builds an abstract space from the start" more than "training sharpens it." The training-time lift shows on the word-position label, 0.60 → 0.66.

The corroboration

The neuroscience agrees, from three directions, and it matters because it means this is not a quirk of our setup.

Numenta, the group whose HTM mechanisms the pure-biology round was built from, themselves dropped HTM and turned to transformers when they moved to language. The people who most believed in the gradient-free cortical model reached for the gradient when they needed abstraction over language.

And two results from the prefrontal-cortex literature land exactly on the split. Lindsay 2017 shows a Hebbian random network reproduces the brain's mixed selectivity (high-dimensional expressive mixing) but not the low-dimensional factorized code that the abstraction probe rewards. Hebbian buys you expressiveness for free; it does not buy you abstraction. Wutz 2018 shows that in the brain, high-level abstraction is literally built by a top-down process: a control operation imposed from above, not a free product of a feedforward stack. Abstraction is constructed, deliberately, by a learning signal. That is the same claim DC makes with a swapped update rule.

The frontier, flipped

The open question has changed shape. It is no longer "is a gradient needed?" Six negatives and one positive have answered that. It is now:

Can a local gradient (a three-factor rule, e-prop, predictive coding, forward-forward) keep the win inside the online, local, bounded regime the program holds itself to?

That is the prize: an online, local, bounded learner that nonetheless does credit assignment well enough to build an abstract space. The global oracle proves the target exists. The next experiment runs a biologically-plausible local gradient on the same harness and asks whether it clears the wall while staying inside the laws. And the corroboration points at where to look. Wutz says abstraction is built top-down, so the local credit signal may want to ride the top-down pathway, fusing the priming track and the credit-assignment track into one mechanism rather than two.

The posts behind it

• The node-architecture concept-boundaries round: surprise carves words, not meaning above them; the runtime that carried the rest.
• The pure-biology HTM round: the back half of the prefrontal arc; sequence and compression, but not abstraction.
• Slower, and from above: temporal pooling and top-down feedback work, and still do not abstract; the fifth and sixth negatives.