The empty cell, filled

Theory update · 2026-06-28 · the empty cell, filled · experiment P0.5

The empty cell gave a partial answer: a sparse code carrying per-unit credit does abstract, climbing to the backprop ceiling, but it does not stay there. It peaks and collapses, because k-WTA bounds how many units fire while the credit pathway still drives the same few to win for every input. The diagnosis named the second cause and the machinery to attack it. P0.5 runs that machinery as an ablation ladder, one piece at a time, and finds the cure. Boosting and positive weights together hold deep transfer abstraction at the ceiling to the last checkpoint, on both seeds, with more than two hundred distinct winner-codes, five times the diversity of the dense reference and the lowest bits-per-char of any arm. Two honest negatives come with it: boosting alone still collapses, and the textbook sign-only credit rule breaks the code outright. The empty 2x2 cell is filled. A sparse substrate holds stable abstraction, and the dense reference is no longer the only thing that does.

The cell, and the second cause

The architecture review found a box the program had never filled: per-unit credit on a sparse code. P0 built it and got a result that was real and partial. The sparse code abstracted, touching the backprop ceiling of 0.484 and beating its no-credit control by a sixth of a point. Then it fell back to the floor.

P0 also said exactly why. The collapse is dual-cause. One cause is the substrate, the dense normalization that pins every code to the same magnitude and throws away the variance, and k-WTA fixes that. The other cause is the credit pathway: a dense feedback signal drives the pre-activations low-rank, the same directions get amplified, the same units keep crossing the top-k threshold, and the winner-sets stop varying across stimuli. The rank-one collapse of the dense code does not disappear on a sparse substrate. It reappears, wearing a different mask, as a winner-set collapse. k-WTA bounds how many units fire. It does nothing about which.

So P0 named the fix: more machinery, aimed at the second cause. Three pieces, each from the brain's own answer to a code that keeps firing the same cells.

• Boosting, the duty-cycle homeostasis from the spatial pooler. Each unit tracks how often it has been winning. A unit that wins too much is held back, a starved unit is lifted into the competition. A winner-set cannot shrink to a handful when every unit is forced to take its turn.
• Positive weights, no bias, the sparse-distributed-memory regime Bricken's continual learner runs in. Clamp the forward weights to non-negative after every update, hold the biases at zero, so a unit's drive is a sum of positive evidence and no learned weight can wire a cell to always win or always lose.
• Sparse-permanence credit, the htm form: instead of the dense feedback-alignment write that touches every incoming weight of a winner, move only the synapses from active inputs to winning units, by a bounded permanence step whose sign follows the credit.

P0.5 is the ablation ladder over these. Start at P0 (no diversity, the collapse baseline), add boosting, add positive weights, add the permanence credit, one rung at a time, and watch which rung, if any, converts the transient peak into a held final. Two anchors bracket the ladder: the dense-with-credit reference (the prior only-stable abstractor) and the no-credit control (the flat floor). Everything else is held byte-for-byte from P0. Same eighteen thousand characters, same two seeds, same probes.

What happened

The cure is two pieces, and it takes both.

The numbers, at eighteen thousand characters, depth three, both seeds shown where they differ: the deep transfer abstraction (CCGP) at the final checkpoint, the deep dimensionality (participation ratio), the count of distinct winner-codes across the stimulus set, and bits-per-char.

Read the ladder rung by rung and three things are clear.

Boosting and positive weights together hold the abstraction at the ceiling. The +BOOST+POS arm reaches a final deep transfer score of 0.485 and 0.489 across the two seeds, at the backprop ceiling of 0.484, and it holds there to the last checkpoint on both seeds, not as a transient peak. Its deep dimensionality is 6.6 and 9.0, far above the rank-one collapse. And the diversity read is decisive: 208 and 225 distinct winner-codes, against P0's three to five. The same handful of units no longer wins for every input; the population spreads. The win comes with the lowest bits-per-char of any arm, 3.74 and 3.77, below even the dense reference. So the empty cell is filled. A sparse substrate, given per-unit credit and the right diversity machinery, holds stable abstraction at the ceiling, with five times the code diversity of the dense reference and better prediction.

Boosting alone is not enough. The +BOOST rung, homeostasis with the dense credit pathway and signed weights, still collapses. Final deep CCGP 0.314, dimensionality at rank one, one or two distinct codes. This is the load-bearing ablation. The mechanism is precise: signed weights let the feedback-alignment credit drive a few units to always win, and boosting on its own cannot overcome a credit signal that keeps pushing the same direction. Clamp the weights non-negative and that always-win route is gone; then boosting keeps the winners spread. The minimal sufficient pair is boosting and positive weights. Neither does it alone.

The textbook sign-only credit breaks it. The +FULL rung adds the htm sparse-permanence credit, the sign-only potentiate-or-depress step, and the code collapses to a single winner-code. The naive sign-only rule saturates the active-to-winner synapses and freezes the code one direction. So this is an honest negative: keep the graded credit, add the diversity machinery around it, do not replace the credit with a sign.

Boosting and positive weights together hold deep transfer abstraction at the 0.484 ceiling to the last checkpoint, on both seeds, with more than two hundred distinct winner-codes against the dense reference's forty, and the lowest bits-per-char of any arm. Boosting alone still collapses. The sign-only permanence credit breaks the code. The cure is two pieces, and it takes both.

What this means

P0 sharpened the architecture review's headline (the collapses are seventy percent a substrate artifact) into a dual-cause collapse. P0.5 sharpens it once more, into its amended, tested form.

The substrate-artifact thesis holds, amended. The simple version (swap the dense code for a sparse one and the collapse dissolves) is false, and P0 already showed it: k-WTA plus the hypersphere support is necessary and not sufficient. The amended version is true: a sparse substrate with per-unit credit holds stable abstraction once two more supports are added, boosting and positive weights, the exact machinery the literature pairs with the sparse continual learner. The collapse was curable. It is cured.

So the dense reference is no longer the only stable abstractor. Through the whole abstraction line, the dense-with-credit stack was the one thing that abstracted and stayed at 0.50 across the run. Now a sparse code matches it, holds the abstraction at the ceiling, and does so with far more diverse codes (200-plus against 40) and lower bits-per-char. The cell that was empty when P0 opened it, and partial when P0 closed it, is filled.

It does not beat backprop. The honest bound is sharp and worth stating twice. The +BOOST+POS arm matches the 0.484 ceiling. It does not exceed it. This is a sparse, gradient-free code reaching the backprop bar and holding there, not passing it. The road past the ceiling, the attention round showed, is a change to the credit pathway itself, and this is not that change.

And the result is eighteen thousand characters, two seeds. The depth-four run, the hundred-thousand-character confirmations, and the text8 bits-per-char plateau test are running. The win is decision-grade at this scale and seed-robust across the two seeds; it is not yet the scaled, multi-seed lock. The +FULL negative is real and stands: the credit stays graded.

The substrate-artifact thesis holds in its amended form: k-WTA and the hypersphere support alone are insufficient (the P0 result), but adding boosting and positive weights makes a sparse code hold stable abstraction at the ceiling. The dense reference is no longer the only stable abstractor. The honest bounds: it matches the ceiling, it does not exceed it; this is eighteen thousand characters and two seeds, with confirmations running; and the sign-only permanence credit is a genuine negative.

Lineage

Grew from the empty cell and its dual-cause diagnosis, which named the second cause of the collapse (the dense credit pathway that k-WTA leaves untouched) and the exact machinery to attack it. It reuses P0's k-WTA sparse stack and its per-unit precision-gated credit verbatim, the same transfer-abstraction, dimensionality, and winner-code probes, and the same eighteen thousand characters and two seeds. It grounds in the spatial pooler's duty-cycle boosting (the anti-collapse homeostasis), in sparse distributed memory (Bricken 2023, the positive-weights, no-bias regime), and in the htm permanence rule (the sign-only negative).

Led to the scaled confirmation queue: the depth-four run, the hundred-thousand-character confirmations, and the text8 bits-per-char plateau test, each checking whether the eighteen-thousand-character win holds at scale and across seeds before the sparse substrate enters the locked core. And to the standing instruction the +FULL negative leaves: diversity machinery around a graded credit, not a sign-only credit that saturates.

Thread: abstraction, and online learning without a global gradient. A decision-grade positive that fills the empty 2x2 cell, holds stable abstraction on a sparse code at the backprop ceiling, and shows the minimal sufficient machinery: boosting and positive weights, together.