What changed: two mechanisms that compose

Theory update. 2026-06-27 · the first composition that holds · experiment SYNTH (E1, E2)

The locked architecture drifts on a long fresh stream: prediction degrades and the deep code collapses toward one direction. Two separate anti-collapse mechanisms have turned up, each a partial fix on a different axis. Lateral inhibition between columns lowers the cost and lifts abstraction above the ceiling, but the deep code still collapses. Interior-altitude replay holds the deep code high-dimensional, but its abstraction fades below the ceiling. This round runs them together on the drift stream. They compose. At three million characters, with both wires on, the deep code stays high-dimensional, stays above the abstraction ceiling at every checkpoint, and stays stable, where each one alone fails on a different count. The first composition in the program that holds. The ten-million-character confirmation since refines that: the composition is a real drift mitigation (the bare config collapses, both wires keep the deep code alive) but a partial one that degrades at scale, with abstraction slipping below the ceiling by seven million. It reduces the drift, it does not cure it.

The question

The drift is the open frontier. The architecture is locked above the backprop ceiling on a bounded corpus, but a ten-million-character run showed it does not stay itself on a long fresh stream. Held-out bits-per-char bottoms near a million characters and then rises, and the deep code collapses toward rank one. The model forgets the held-out distribution as it adapts to ever-new text, and the deep level says the same thing no matter the input.

Two mechanisms have since pushed back on that collapse, from two directions. The first is lateral inhibition: columns at the same altitude see and suppress each other, so the winning column varies with the stimulus and the population recovers the variance the stack had pinned away. The second is interior-altitude replay: a stored configuration is re-fired and reinstated at a middle level, Minsky's K-line at the altitude his theory predicts. Each was measured alone, and each was honest about its limit. Inhibition lifts prediction and abstraction but did not clearly hold the deep code up. Replay holds the deep code up but its abstraction tied a weaker control below the ceiling.

So the question is the one both posts ended on. The two wires fight the same rank-one collapse on different axes, one across columns within a level, one across time at an interior level. Do they compose? The program has a sharp reason to doubt it. The capstone combine (the run we called A1) tried to add two levers, breadth and normalization, and they did not add, because they act on the same axis of variance and concatenation cannot undo a per-column collapse. A composition is not free. It has to be shown.

What we tried

We ran four arms on the exact drift stream where the locked config drifts: three million fresh characters of text8, the locked depth-three config, three diverse views per level, one seed. The arms differ only in which anti-collapse wire is on.

• baseline. No inhibition, no replay. The locked config on the fresh stream.
• inhibition. Lateral divisive inhibition between the same-level columns, and nothing else.
• interior replay. Re-fire a buffered configuration and consolidate it at the interior level, and nothing else.
• both. Inhibition and interior replay together.

Running inhibition at three million characters needed a faster substrate. The original inhibition stack ran at about 174 characters a second, hours per arm. So the divisive inhibition rule was ported into a compiled multi-column stack that runs at about 2,100 characters a second. The port earns its trust through a gate run before any real arm: with one column and inhibition off, it reproduces the locked single-column learner bit for bit on bits-per-char, and the drift runner at one column reproduces the earlier interior-replay baseline. So the fresh hot path is the locked math, unchanged, and the only new things in play are the two wires. The headline reads are held-out bits-per-char, the apex participation ratio (the dimensionality of the deep code, the anti-collapse read), and the best transfer CCGP (abstraction on held-out text), each at three checkpoints across the stream. The backprop CCGP ceiling is 0.484.

What happened

The deep code's dimensionality and its abstraction are the two tests. Read the final checkpoint first, then the trajectory.

Read it as the two single-mechanism failures and the one composition.

Inhibition alone loses dimensionality. It lowers held-out bits-per-char at every checkpoint (a mean of about 0.15 below the baseline) and pushes abstraction above the ceiling, peaking at 0.566. That is the bounded-corpus win carried to scale. But the deep code still collapses: its apex dimensionality dips to 1.00 at the two-million mark, rank one, and the stability flag flips across checkpoints. Inhibition fixes prediction and the quality of abstraction. It does not, on its own, hold the deep code up.

Interior replay alone loses abstraction. It holds the deep code high-dimensional all the way, apex never below 3.13, and it is stable throughout. But its abstraction fades: best CCGP runs 0.571, then 0.346, then 0.292, ending well below the ceiling. Replay protects the deep code's dimensionality and its prediction. It does not, on its own, keep the abstraction.

Both together holds all four. With inhibition and interior replay on, the deep code stays high-dimensional at every checkpoint (apex 5.80, 4.46, 3.75, never collapsing), stays above the 0.484 ceiling at every checkpoint (best CCGP 0.560, 0.596, 0.561; at the two-million mark the per-level reads are 0.596, 0.565, 0.490), stays stable throughout, and keeps bits-per-char bounded. Each mechanism covers the other's failure. Inhibition supplies the abstraction that replay lets fade; replay supplies the dimensionality that inhibition lets collapse. The composition holds where neither half does.

The lesson

The two anti-collapse mechanisms compose. Inhibition alone loses dimensionality (the deep code collapses to rank one at two million characters); interior replay alone loses abstraction (its score fades to 0.292, below the ceiling). Together they cover both failure modes: the deep code stays high-dimensional (apex 3.75 to 5.80), above the 0.484 ceiling at every checkpoint (0.560 to 0.596), stable, with bounded bits-per-char. This is the first composition in the program that holds, at this scale. The ten-million confirmation below shows the hold is partial and degrades on a longer stream.

A1 is the foil. There, two levers acted on the same axis of variance, and concatenating them added nothing. Here the two wires act on orthogonal axes: inhibition reaches across columns within a level, replay reaches across time at an interior level. Because the axes are different, the fixes complement instead of redundantly stacking. The drift was the open frontier, and it now has a candidate fix made of two mechanisms that cover each other's blind spot.

The honest caveats

• Three million characters, one seed. This is a trimmed run on a single seed, the same scale at which the baseline already drifts, not the full ten million where the drift was first found, and not yet multiple seeds.
• The trajectory is the signal, not any one point. The single-checkpoint CCGP and dimensionality reads are noisy: on the single-mechanism arms the apex dimensionality bounces between 1.0 and 5.8 across adjacent checkpoints. What is robust is the pattern across the three checkpoints, where both stays up and above the ceiling while each half drops on a different axis. No single number carries the claim.
• The drift is tamed, not perfectly flattened. With both wires on, bits-per-char still ticks up at the last checkpoint, 3.654 to 3.818. The composition turns the collapse back; it does not erase the non-stationarity.
• Confirmation ran, and it qualifies the result (see the next section). A ten-million-character run came back. The composition holds at three million, but at ten million it is partial: it mitigates the drift without curing it. This is not yet locked into the architecture.

Confirmed at scale, with a caveat

The confirmation ran: ten million characters, seven million of fresh training against a fixed three-million held-out slice, seed zero, the same baseline against the same both. It changes the reading. The composition is a real drift mitigation, and it is partial, and it degrades as the stream gets longer.

The baseline, given the full stream, does what the drift always does, only more so. Held-out bits-per-char bottoms at 3.585 near four million characters, then climbs to 4.120 by seven million. The deep code's dimensionality slides from 3.61 down to rank one (its participation ratio reads as undefined at the end, the degenerate-code signal), and abstraction fades to 0.317, well under the ceiling. The flag reads unstable at seven million. Left alone, the locked config collapses.

With both wires on, the model survives. At seven million it beats the collapsed baseline on every axis: bits-per-char 3.694 against 4.120, deep dimensionality 1.83 (alive, never the baseline's rank-one collapse), abstraction 0.429 against 0.317, stable against unstable. So inhibition and interior replay together are genuinely a drift mitigation: without them the deep code dies, with them it lives.

But the protection is weaker than it was at three million. The deep dimensionality ends at 1.83, where three million held it near 3.75. Abstraction ends at 0.429, below the 0.484 ceiling, where three million sat near 0.56 above it. Bits-per-char bounces as high as 4.24 mid-stream before settling. The clean above-ceiling hold of the three-million run is not retained at seven million. The drift is reduced, not erased, and the protection thins as the stream lengthens.

So the honest verdict is partial mitigation, not a cure. The mechanisms help exactly where the drift bites, and they keep the deep code alive on a long fresh stream where the bare config collapses. They do not yet hold it above the ceiling at scale. Two caveats stand behind the numbers: a single seed with single-checkpoint reads, so the seven-million endpoint is the comparison and the mid-stream bounces are noise around a degrading trend, not signal; and the replay buffer (200,000 samples, refresh rate 0.05) was carried over from the three-million run and may be undertuned for seven million characters of fresh stream. A larger reservoir might hold the protection longer. That is untested.

The frontier

The deep collapse that capped the architecture at depth three looked structural, and it is not. One mechanism reopened it across columns, a second reopened it across time, and the two together keep the deep code alive across a long fresh stream where each alone, and the bare config, all collapse. What the ten-million run settles is the size of the win: it is a mitigation, not a cure. The drift is reduced, the deep code survives, and abstraction still slips below the ceiling as the stream runs on. The open question is now sharper: hold the protection above the ceiling at scale, not merely keep the code alive. Two candidates are in hand and untested: tune the replay reservoir for a longer stream, and run more seeds to settle the noise. Two small wires (columns that suppress each other, and a memory reinstated at the right altitude) turn the locked architecture from collapse-prone into something that survives a long fresh stream. Holding it above the ceiling there is the next run.

Lineage

Grew from lateral inhibition clears the collapse, the across-column anti-collapse mechanism whose scale claim this confirms in part and completes; from replay at an interior altitude, the across-time mechanism this composes with; and from the architecture, locked above the ceiling, whose capstone combine showed two levers on the same variance axis do not add, the foil this composition clears.

Thread: the collapse that caps abstraction, and the drift that caps the architecture on a long fresh stream. The two mechanisms are cortical lateral inhibition and Minsky's interior K-line. The frontier is continual learning under a long non-stationary stream, with a candidate fix now in hand and a confirmation run pending.