Three swings after the keystone

2026-06-27 · three swings after the keystone · experiments DF, DG, DH

DE built the keystone: per-unit credit ridden down a top-down apical pathway, gated by precision, abstraction climbing with depth. It named three next swings, and we ran all three. A label-attractor (DF) compresses dimensionality harder than anything yet and does not abstract, an honest negative that rules out a tempting shortcut. Different attended views (DG) make voting earn its keep, confirming Monty's diversity requirement and winning prediction, and they do not abstract: prediction and abstraction factor apart. And scaling the apical fusion (DH) buys one extra altitude of depth, a three-level rising code that nearly touches the backprop ceiling, then saturates, and the precision gate that was the hero at two levels flips into the chief destabilizer at depth. The next wall has a name: stability.

Where this picks up

The apical fusion was the third positive and the architectural keystone: the first stack in the program whose abstraction score rose with depth. It also left a to-do list. Three swings, each a different question DE could not answer on its two-level harness:

• Is there a gradient-free shortcut to abstraction, a self-supervised label that builds the abstract level without per-unit credit at all?
• Does diverse voting, many columns each attending a different view, abstract on its own, or is it a different job?
• Does the fusion scale, more depth, more data, and how far does abstraction climb?

DF, DG, and DH answer these in turn. Two are honest negatives in the useful sense; the third is a mixed verdict that names the next wall.

DF: the label that compresses but does not abstract

Barrett's theory of constructed emotion makes a sharp claim: a shared label (a word) is the stabilizing variable that creates an abstract category over wildly variable instances. If that were the mechanism, it would be a gift: a self-supervised attractor, no error signal, no credit assignment, fully gradient-free. So DF runs it as the most direct test the regime allows.

On DC and DD's harness, unchanged, a new WORD-ANCHOR arm adds a label-anchored prototype pull on top of the Oja step: a local pre·(prototype − post) update toward a shared per-word prototype, the prototype a leaky running average of its members' codes, with a bounded dictionary. No readout error, no backward pass. The label is the current word identity. This is the word stabilizing the level below, exactly as Barrett describes.

It does not abstract.

The prototype pull is a powerful dimensionality sink: participation ratio collapses to 5.8, below even the no-credit Hebbian floor of 17, the most aggressive compression in the whole line. And the abstraction score goes the wrong way: transfer sits at 0.321, below both the raw-input baseline (0.339) and the Hebbian control (0.335); the stronger word-position label collapses to 0.249 against a 0.503 baseline; bits-per-char never drops. It is worst exactly where it matters most (on held-out transfer, true generalization, and on the stronger label) and it stays there across every anchor learning rate we tried. This is Barrett's own compression-is-not-abstraction caution, reproduced in silico: pooling variable instances onto one word handle shrinks the space without building the cross-context code the abstraction probe rewards.

We held the no-label-leak guard carefully. Within a word like "cat," the c, a, and t share one anchor, so anchoring mixes the probe classes (vowel/consonant, word-position) under a single handle. It structurally cannot copy a probe label, which means the absence of an abstraction rise is a real negative, not a guard against an inflated one.

A self-supervised label attractor compresses dimensionality harder than any mechanism in the line and does not build an abstract space; it scores below the raw baseline on transfer. The gradient-free route to abstraction is per-unit credit (DD/DE), not a label that pools instances onto a word.

There is a reconciliation worth keeping. The word-prototype may be the right target; it just lacks a way to deliver it. Used as a direct attractor, it collapses; ridden down DE's per-unit apical credit pathway, as a top-down label-target rather than a running-average attractor, it becomes a target-propagation flavor worth a future experiment. The label may yet have a role; as a standalone attractor, it does not.

DG: different views make voting earn its keep

The second swing tests a mechanism from a different tradition. Monty, the Thousand-Brains voting model, insists that columns voting only help when the columns see different things; identical sensors voting buy nothing. DE named this the highest-value test. DG runs it cleanly, and the result factors the architecture into two jobs.

A set of cheap count-table columns, each with a calibrated confidence, each reading a different attended view: raw character spans at spread offsets and widths, plus a content cue keyed to word-position. They pool as a confidence-weighted product of experts, strictly online. Four arms: a SINGLE column; SAME-VIEW (many identical windows, the degenerate vote); DIFF-VIEW (many different views); and DIFF-NOPREC (different views, precision weighting off).

Four findings, each tied to the numbers.

Different views beat identical views decisively, at every scale. SAME-VIEW lands at 5.108 bits-per-char, worse than a single column, because a product of experts over many identical distributions over-sharpens, the exact saturation an early voting experiment warned about. DIFF-VIEW sits a full one to two bits better, with decorrelation rising from zero to 0.444. Monty's diversity requirement is confirmed empirically: voting earns its keep only when the voters disagree.

But diverse voting beats a single column only when the data is thin. At ten thousand characters DIFF wins by half a bit; the gap closes and reverses as data grows, and by two hundred thousand the single long-context column wins by 0.07. Diverse short and cue views fill in a starved predictor, then dilute a well-fed one.

Precision-weighting is essential. Turn it off and the pool blows up to 8.5 bits: cold, unseen views vote at full confidence. Attention-as-precision does real work.

And abstraction does not move. DIFF leaves the transfer abstraction score flat at about 0.49 and raises dimensionality from 10 to 32: more expressive, not more abstract, the same signature the pure-biology and temporal-pooling negatives showed.

Different attended views make voting earn its keep (Monty's diversity requirement, confirmed) and they win prediction in the data-starved regime. They do not abstract. The architecture factors cleanly: prediction comes from diverse, precision-weighted views and voting; abstraction comes from per-unit precision-gated apical credit. Two jobs, two mechanisms, sharing one precision signal.

DH: depth buys one altitude, then stability is the wall

The third swing scales the keystone itself. DE ran on two levels with hand-tuned stability and asked the obvious question: does abstraction keep climbing with depth, and does more data help? We ported the apical fusion to the faster MLX substrate (an L-level stack with a fixed-random apical credit wire between every adjacent pair, gated by the precision burst, DE's design and stabilization carried over verbatim) and swept depth and data.

It first reproduced DE on the new substrate: the gated fusion's L2 transfer (0.424) beats single-layer feedback alignment (0.392) beats the raw baseline (0.324), climbing from L1 to L2. Ordering and rising altitude confirmed before scaling. Then the sweep, and an honest mixed verdict.

Depth buys one extra altitude of abstraction, then saturates. The depth sweep climbs cleanly to three levels:

Depth three is the first genuine three-level rising-CCGP code in the line: its apex (0.465) nearly reaches the backprop ceiling (0.484), robust across a second seed (0.436) and a wider hidden layer (0.416). But depth four adds no abstraction (about 0.47, the same as three) and starts drifting, its apex dimensionality collapsing to near three, a degenerate-code artifact we flag rather than count.

More data does not help; it raises the risk of blowing up. DE was stability-limited, not data-limited. At the carried-over learning rate the depth-three stack diverges past about twenty thousand online steps: at fifty thousand, bits-per-char runs from 3.98 to 30.5, the unbounded-forward-activation failure DE warned would worsen with more apical wires. Lowering the learning rate restores stability but the abstraction score plateaus: a hundred thousand characters, stable, lands L3 at 0.370, no higher than eighteen thousand reached.

And the precision gate flips sign. At two levels precision-gating was load-bearing (DE). At depth it is the primary destabilizer: the gated arm diverges at depth three and four, and once tamed enough to stay alive its benefit vanishes (L3 0.256, no rise). So at depth the ungated apical wire is the operative mechanism, the opposite of DE's two-level finding. One precision read three ways still holds as a principle; one of those reads just turned dangerous at scale.

Scaling the apical fusion buys one real extra altitude of abstraction (a three-level rising code whose apex nearly touches the backprop ceiling) then it saturates, and more data only raises the risk of divergence. Stacked feedback alignment is not stable at scale with the carried-over knobs: more wires plus more steps drive forward activations unbounded, and the gate that was the hero at two levels becomes the chief destabilizer at depth. The next bottleneck is stability, not depth or data.

What the three swings add up to

Read together, DF, DG, and DH do something cleaner than any one of them alone: they finish the diagnosis the keystone started.

DF rules out the one tempting gradient-free shortcut (a label attractor) and sends us back to per-unit credit as the only route that abstracts. DG factors the architecture in two, proving that diverse voting is a prediction organ and abstraction lives elsewhere, on the per-unit credit pathway. And DH takes the credit pathway to scale and finds its ceiling (three levels, near the backprop mark) and its wall: stability. The forward activations grow unbounded; the precision gate flips; the deep code collapses even where the stack stays alive.

That is the whole setup for what comes next. The abstraction architecture works and reaches the backprop ceiling at depth three. To push past it, the program does not need a new idea about credit; it needs to stabilize the stack so depth and data stop hurting. That is a parameter-sweep question, and it is exactly the round that follows.

Lineage

Grew from the apical fusion, whose harness, probes, and mechanism all three swings reuse, and whose named next steps they run: the Barrett label probe (DF), multi-attention (DG), and scaling on the MLX substrate (DH). DF grounds in Barrett's constructed-emotion theory and its own compression-is-not-abstraction caution; DG in Monty's diversity requirement and the offset-attention and cue-retrieval views it pools; DH in the stability caveat DE recorded, now the operative bottleneck.

Led to the sweep round: four parameter sweeps that attack DH's stability wall head-on and lock the architecture above the ceiling.

Thread: abstraction, and online learning without counting. Two honest negatives and a mixed scaling verdict that together name the next wall (stability) and hand the sweep round its target.