Where we are

Theory update · 2026-06-28 · where we are · the two engines, the wins, and the wall

Stand back from the run of experiments and the shape is clear. The program has built two engines. A prediction engine reads the next character at the n-gram level, near 2.1 bits-per-char, count-native and gradient-free. An abstraction engine builds a transfer-abstract code at or above the 0.484 backprop ceiling, with no gradient at all, which is the genuinely novel result. Each is strong on its own axis. The recent rounds were the wins that got them there: P0.5 cracked the winner-collapse into stable sparse abstraction, grow-then-prune broke the bits-per-char plateau, BLEND-NORM corrected the prediction calibration, and a gentler learning rate found a lower floor, while the attention and working-memory subsystem closed its own arc. The honest scorecard: on prediction we are a mile from a strong transformer, not a hundred light-years, the n-gram level and far from SOTA but in the right universe; the abstraction result is the real prize. The deep open problem is that the two engines do not fuse: one code cannot be both abstract and count-reusable, the rank/reuse tension, with a stable 0.45 bits-per-char tax to prove it. The bet to resolve it is the cortex's own answer, two coupled laminar layers that share information rather than one representation. And what comes after is the larger architecture: generation and production, learning by example.

Two engines, each strong on its own axis

For months the work read like a single line chasing a single number. It is not. The program has converged on two engines, and the cleanest thing to say about where we are is to say what each one does and what it does not.

The prediction engine is the diverse-view count bank. Many count columns, each reading a different view of the recent context, each a small calibrated conditional table, pooled by the BLEND-NORM geometric mean and grown with data by grow-then-prune. It predicts: on text8 it reaches held-out bits-per-char near 2.1, the n-gram level, and it keeps dropping as the bank grows. It is online, single-pass, gradient-free, bounded. It combines evidence by reuse: a context key seen before accumulates counts, and diverse views vote.

The abstraction engine is the sparse apical stack. A k-WTA sparse code with boosting, positive weights, and per-unit precision-gated apical credit carried down a real top-down wire. It abstracts: deep transfer abstraction (CCGP) at or above the 0.484 single-layer backprop ceiling, about 0.485 to 0.52, stable and seed-robust, with hundreds of distinct winner-codes. And it does this without a gradient. That is the part worth pausing on. The whole abstraction line started from six different gradient-free mechanisms that each carved structure and compressed dimensionality but never built an abstract, factorized, generalizing space. The verdict looked like a closed door: abstraction needs a gradient. The program then walked it open, one experiment at a time, to a biologically-shaped, online, local, bounded learner that builds an abstract code with no gradient at all. That is the genuinely novel result of the program, and it is the abstraction engine.

Each engine is strong on its own axis. Neither does the other's job, and that is not a failure of effort; it is the finding.

The wins that got us here

The recent rounds were the wins that turned two hand-tuned, fragile results into two settled engines.

P0.5 cracked the abstraction-wall winner-collapse. A bare sparse code carrying per-unit credit abstracts to the ceiling and then collapses: the credit pathway drives the same few units to win for every input, so the high-rank code degenerates to a handful of winner-sets. P0.5 ran the diversity machinery the brain uses, as an ablation ladder, and found the cure is two pieces and takes both. Boosting and positive weights together hold deep transfer abstraction at the 0.484 ceiling to the last checkpoint on both seeds, with more than two hundred distinct winner-codes against the dense reference's about forty. Boosting alone still collapses; the minimal sufficient pair is both. So a sparse substrate carrying per-unit credit holds stable abstraction, and the dense reference is no longer the only thing that does (the empty cell, filled).

Grow-then-prune broke the bits-per-char plateau. The fixed twelve-column voting bank improves to about a million characters and then flatlines (2.234 to 2.112 to 2.109), while a plain n-gram keeps falling because it keeps growing. The growable bank adds a longer-context column when the held-out gain stalls, caps the count, prunes the least-used, and goes 2.004 to 1.875 at ten million where the fixed bank stays at 2.109, within 0.047 of the n-gram floor. The plateau was a fixed-capacity artifact, and growth dissolves it by doing what the n-gram does, under a budget (growth breaks the plateau).

BLEND-NORM corrected the prediction calibration. A test of Global Workspace ignition surfaced a calibration fix the engine needed. The old pool was an un-normalized product of experts, raising the consensus to the power of the summed confidences (an effective exponent about 3.3 over ten active columns), which over-sharpens and inflates the bits. A normalized blend that keeps every column at the right temperature reads a full bit better and beats hard winner-take-all at every scale. It also corrected an inflated absolute bpc: the experiments built on the old pool read about one bit high (the vote was too loud).

The learning-rate sweet spot. Re-reading a bounded slice helped prediction a little and then flattened, and that plateau looked like a ceiling on repetition. It was the rate. The locked rate (0.005) crams the slice on the first pass and has nothing left to compound; drop it to 0.001 and repetition keeps paying across all ten passes to a lower floor, 3.637 against the aggressive 3.762. The optimum is a U, and the rate that proved too hot is the apical stack's global rate, so the implication reaches past re-reading (gentle learning compounds).

The attention and working-memory subsystem. A parallel arc, AT1 through AT3 and a bounding fourth, made the per-column scan a learned action: columns learn where to look (with lateral inhibition on the action space), a higher level guides the scan by context, and a basal-ganglia-style gate decides when to update the held theme. The full stack beats hand-set fixed views, online and gradient-free. It is a prediction and working-memory win, not an abstraction one, the prediction side of the two-engine split (a gate for working memory, attention is prediction, not abstraction).

The honest scorecard

Two axes, two honest reads.

On prediction, we are a mile from a strong transformer, not a hundred light-years. The growable bank reaches about the n-gram floor (1.875 against 1.829), it clears the no-skill ceiling (4.755) comfortably, and it is competitive at small data. But it is the n-gram level, about 0.85 bits-per-char behind text8 SOTA near 1.0, and a strong transformer reads 1.08 on the same corpus, mostly because of the regime (epochs plus backprop against our single online pass). We are far from the frontier, in the right universe. And prediction is not the program's headline axis; the engine is honest about being a count predictor that keeps learning from data instead of saturating.

On abstraction, the result is the real prize. A gradient-free, online, local, bounded learner that reaches transfer CCGP at or above the single-layer backprop ceiling, stable and seed-robust, is something the gradient-free literature did not have. That is where the program is genuinely ahead of where it started, and ahead of the closed doors that came before it. (The 0.484 mark is the single-layer reference; a multi-view backprop is the fair ceiling to recheck before any claim against deep backprop, and we have not made that claim.)

The deep open problem: the rank/reuse tension

The two engines are each strong, and they do not fuse into one code. That is the central open problem, and it is now named precisely.

Put a count-native prediction head on the abstract code, against the same count machinery on the raw context, on the same frozen substrate. The count head extracts real signal from the abstract code: it beats the stack's own readout by 0.8 to 1.0 bits-per-char, so one substrate can feed both heads. But it stays about 0.45 bits-per-char behind raw context, and the gap holds stable across 5.5 times the data, while the same code's transfer CCGP is 0.499, above the reference. The same code abstracts well and predicts worse (the rank-reuse tension).

The mechanism is a tension in the representation itself. The abstract code is nearly unique per position (three to six counts per key even at seventy thousand positions), because the high rank that makes it abstract makes it almost never repeat, and a count table predicts by reuse. So one code cannot be both abstract (high-rank, unique per position) and count-reusable (must repeat). The rank a representation needs to abstract is the rank that makes it too unique to count. That is why the program keeps drawing two engines, and it is the thing in the way of a single robust, scalable model.

The bet to resolve it: coupled laminar layers

If one code cannot be both, keep both codes and couple them, the way the cortex does. This is the bet, and it is sourced from the canonical cortical microcircuit, not invented.

The literature returns a clear verdict: the tension is not a bug to optimize away, it is why the cortex uses different layers. Bastos 2012 maps the two jobs onto layers and bands, superficial L2/3 carrying feedforward prediction-error and deep L5/6 carrying feedback prediction, distinct populations with distinct jobs, the external license to stop merging the two codes. Larkum 2013 gives the cellular warrant: one L5 pyramidal cell binds bottom-up (basal) and top-down (apical) streams by coincidence, without merging them, because they enter different compartments. The abstraction engine's per-unit precision-gated apical credit is already a Larkum cell; the program simply never used the apical pathway to carry a second engine's signal.

So the design keeps both codes and shares information, not representation, across two seams: a coarse abstract cluster-id (coarse enough to repeat) appended to the prediction engine's context key, and the prediction engine's pooled code fed into the abstraction stack's basal input. The cheap first slice, count-only, is running now: does a coarse abstract tag inform prediction without simply fragmenting the counts? The honest limit, stated by the sources, is that a predictive-coding circuit is a local gradient and does not promise to climb past the 0.484 ceiling; what it promises is the coexistence the shared code could not reach, low bpc and high CCGP at once. If sharing information helps where merging representation failed, the coupling is real. The full design and roadmap now live on the architecture page.

What comes after

Resolving the tension is the near wall. Past it is the larger architecture the program has always been pointing at.

Generation and production. The internal half is real: a birdsong loop that babbles, re-comprehends its own output, and corrects toward the meaning it intended, with no listener, and a separate learned inverse that produces real dialog words and holds a live, glimmering conversation. The external half is the named next move: a referential game that fixes a shared convention, scored on whether a listener recovers the referent, the anchor against private-code drift.

Learning by example. The whole program is online, gradient-free, bounded, and guided by human cognition. The two coupled engines are the substrate a learner that grows from examples needs: a predictor that keeps learning from data, an abstractor that generalizes, and a way to couple them. That is the road from two strong engines to one architecture.

The honest stance

This is not AGI, and we will not pretend it is. It is a real, validated, online, inspectable, brain-grounded research program. Two engines, each strong on its own axis, one of them a genuinely novel gradient-free abstraction result; an honest scorecard that puts prediction a mile behind a strong transformer and the abstraction result out front; one deep open problem named precisely; and the cortex's own answer being tried as the bet to resolve it. A closed door is information too, and so is a wall you can name.

The posts behind it

• The rank-reuse tension: the two-head experiment that named the central open problem, one code cannot be both abstract and count-reusable.
• One stack, two halves: the round that split the program into two engines and found abstraction composes while prediction does not transfer via the vote.
• The empty cell, filled: P0.5, the sparse code that holds stable abstraction at the ceiling with boosting and positive weights.
• Growth breaks the plateau: grow-then-prune, the growable bank that keeps dropping where the fixed one flattens.
• The vote was too loud: BLEND-NORM, the calibration fix for the prediction pool.
• Gentle learning compounds: the learning-rate sweet spot, a U-shaped optimum with the locked rate too hot.
• The architecture spec: the standing v2 description of both engines, the dimensions, the tension, and the coupled-laminar next direction.