What changed: abstraction that climbs
Theory update · 2026-06-27 · abstraction that climbs · experiment DE
The line ran: DC said the wall was credit assignment, not topology. DD said a per-unit local signal clears it where a broadcast scalar cannot, and named the prize: carry that per-unit credit down a real top-down pathway, the way the brain is thought to build abstraction, gated by precision for when it fires. DE builds exactly that on a two-level stack. The full fusion (apical per-unit credit plus a precision gate) reaches a transfer abstraction score of 0.501, beating single-layer feedback alignment (0.408) and the raw baseline (0.339), approaching the backprop ceiling (0.537). And it is the first stack in the whole program whose abstraction score rises with depth. The priming pathway and the credit pathway turn out to be one mechanism.
What this round was
For months abstraction was the wall. Six gradient-free mechanisms carved structure and compressed dimensionality and never built an abstract space. DC isolated the cause: swap only the update rule and the gradient builds the abstract code, so the missing ingredient is credit assignment. DD made it constructive: a per-unit local signal (feedback alignment) clears the wall while a broadcast neuromodulatory scalar does not, so the diagnosis sharpened to per-unit credit assignment, and a shippable class of learner came into reach.
But DD ran on one flat layer. The neuroscience kept pointing somewhere specific: Wutz 2018 says abstraction is built by a top-down process, an operation imposed from above, not a free product of a feedforward stack. The program had wanted a multi-level architecture with top-down priming since the start. DE asks the constructive question that ties all of this together: carry the per-unit credit down a real top-down (apical) pathway, the same wire that primes the level below, and gate it by precision. Does abstraction survive that delivery, and does altitude finally start to mean something?
The experiment
A two-level stack on the DC/DD harness, unchanged. Input sparse code → L1 hidden → L2 hidden → next-character prediction. Strict online: one character at a time, single pass, batch of one. The same three probes, imported verbatim, now run at both levels, within-slice and on held-out transfer text.
The new wire is the point. L2 sends a per-unit top-down signal down to L1, carrying credit, feedback-alignment style: a fixed random projection, no weight transport. So the top-down pathway that primes L1 and the pathway that assigns it credit are literally the same mechanism. And the magnitude of that top-down credit update is gated by precision, the three-factor "when," a non-negative surprise burst that decides how hard the credit lands.
Five arms differ only in how L1 and L2 learn:
- HEBB-2L, Oja both levels, the control.
- FA-1L, DD's single-layer feedback alignment, the baseline to beat.
- APICAL-FA, per-unit top-down credit L2→L1, ungated.
- APICAL-FA+PRECISION, the full fusion: apical per-unit credit, gated by precision.
- GRAD-2L, backprop both levels, the ceiling.
What happened
The full fusion abstracts, and for the first time in the program its abstraction climbs with altitude.
| arm | bits-per-char | L1 transfer (v/c) | L2 transfer (v/c) | altitude (L2 − L1) |
|---|---|---|---|---|
| raw-input baseline | n/a | 0.339 | 0.339 | n/a |
| GRAD-2L (ceiling) | 3.95 → 3.35 | 0.528 | 0.537 | +0.009 (flat) |
| HEBB-2L (control) | ~flat 4.41→4.47 | 0.335 | 0.459 | (probe artifact) |
| FA-1L (baseline) | 4.14 → 3.80 | 0.408 | 0.408 | flat by construction |
| APICAL-FA (ungated) | 4.14 → 3.95 | 0.248 | 0.355 | +0.107, below baseline |
| APICAL-FA+PRECISION | 4.14 → 4.01 | 0.323 | 0.501 | +0.178 |
Three findings.
Per-unit credit survives delivery down a real apical pathway, but only when it is precision-gated. The full fusion's L2 transfer score reaches 0.501, beating single-layer feedback alignment (0.408), the ungated apical wire (0.355), and the raw baseline (0.339), and approaching the backprop ceiling (0.537). So the top-down priming pathway and the credit pathway can be the same mechanism. Wutz's claim, abstraction is built top-down, holds literally, in our harness, as a working wire.
Precision-gating is load-bearing. Gated reaches 0.501; ungated reaches 0.355. Without the three-factor "when," apical delivery actually underperforms the single flat layer. The "when" of precision adds real value on top of the per-unit "what" of feedback alignment. They are not two nice-to-haves; the fusion needs both.
Altitude finally means something. Both apical arms show the abstraction score genuinely climbing from L1 to L2: the full fusion by +0.178 on the value/context label and +0.117 on word-position (transfer L1 0.499 → L2 0.616). This is the first stack in the program to produce abstract-code-by-altitude, the higher level more abstract than the lower, the exact thing every prior feedforward and Hebbian stack failed to do. Feedback alignment alone is flat by construction (one layer); the backprop ceiling abstracts but does not climb here. Only the apical fusion makes depth pay in abstraction.
The verdict
Carry per-unit credit down a real top-down apical pathway, gate it by precision, and abstraction survives, beats single-layer feedback alignment, approaches the backprop ceiling, and rises with depth for the first time. The priming pathway and the credit pathway are one mechanism. The abstraction wall now has a constructive, biologically-shaped architecture.
This is the third positive of the line and the architectural keystone. DC said what was missing. DD said how to supply it locally. DE puts it on the structure the program always wanted, a multi-level stack with top-down priming, and shows that the priming and the credit can be the same projection, read for when by precision. The "one precision read three ways" principle falls out: a single precision signal sets attention/vote gain, working-memory eligibility, and per-unit credit amplitude. One read, three jobs. The architecture that abstracts is the architecture that primes, and they were never two systems.
The honest caveats
We keep them in full, because the claim is the strongest the abstraction line has made.
- The headline is the full fusion, not apical delivery on its own. Ungated apical (0.355) sits below single-layer feedback alignment (0.408). The top-down wire only earns its keep once precision gates it. The win belongs to the combination, and we say so.
- It needed a stabilized learning rate. A three-wire feedback-alignment stack (readout plus L1→L2 plus the apical wire, with unnormalized deltas) diverges at DD's learning rate over about twelve thousand online steps (bits-per-char blows past 20), and the precision gate amplifies the runaway. The fusion ran only after a per-step update-norm cap on the apical wire (applied identically to both apical arms, for fairness), a gentler learning rate, and a modest burst cap. The result is real and robust across both seeds, but it is not a rule that ran on the first try, and stability at scale is unproven.
- HEBB-2L's apparent rise is an artifact. The control's L2 looks higher than its L1, but its L2 dimensionality collapses to near one, a degenerate code that flatters the probe. That is not abstraction; it is a measurement trap, flagged so it is not mistaken for the real climb.
- Still one architecture family, one probe. The context-window MLP and the CCGP abstraction score. The rising-altitude effect and the precision-gate benefit replicate across two seeds, but this is one architecture, not the full stack, and not yet at scale.
The frontier, reshaped
The abstraction wall is no longer a wall. It is a target with a working approach on it. The open questions are now about scale and robustness, not possibility:
Does the fusion hold at multi-level, multi-column scale on more data, and can the stacked feedback-alignment wiring be made stable without the hand-tuning DE needed?
The named next swings: scale the fusion on the faster substrate to more levels and more columns; add multi-attention (a different attended view per column, the same precision signal read a fourth way) which the attention work independently flags as the highest-value test; harden the stacked-feedback stability so it does not need a gentle learning rate to stay alive; and bring in a complementary, fully gradient-free abstraction probe to check the result from a second direction. Remaining work is scale, multi-attention, and stability. Whether a biologically-shaped, online, local, bounded architecture can build an abstract code is no longer the question. DE answered it: it can, and it climbs.
The posts behind it
- The head-to-head that found the missing ingredient: swap only the update rule; the wall was credit assignment, not topology.
- A local signal clears the wall: the step before this one, per-unit feedback alignment abstracts where a broadcast scalar fails, and names the apical pathway.
- Slower, and from above: the fifth and sixth negatives, and the top-down idea this round finally makes pay.