The current canonical spec, v2

The architecture

The standing, versioned description of what the kinogaki-cortex code currently is: the model the experiments have converged it into. Now two engines: a prediction engine near the n-gram level, and an abstraction engine above the backprop ceiling with no gradient, which do not fuse into one code. The dimensions, the rank-reuse tension, and the coupled-laminar next direction. Re-versioned when the architecture changes; prior versions kept, with a changelog.

Architecture v2 · 2026-06-28 · the current canonical spec

This page is the architecture spec: the model the experiments have converged the code into, described in detail and explained. It is the counterpart to the lab notebook: the notebook is the running journal of every experiment; this page is the standing answer to "what is the kinogaki-cortex code, right now, and why is it shaped this way?" The architecture is now two engines: a prediction engine that reads the next character at the n-gram level, and an abstraction engine that builds a transfer-abstract code above the backprop ceiling, both online and gradient-free. They are strong on different axes and, on the evidence so far, they do not fuse into one code. The whole spec below is organized around that fact. Experiments drive this page; when an experiment shifts the architecture, this spec is re-versioned and the change is logged at the bottom. The full v1 spec is kept at /architecture-v1/.

Overview

kinogaki-cortex is a brain-inspired model of text. It learns the way a person reads: online (every character teaches it, one pass, no retraining), gradient-free in regime (it learns on arrival without a global backward pass over the data), bounded-memory (it lives under a budget and must generalize to stay good), and guided by human cognition at every fork. Those are the four laws; they are not style, they are the constraints the whole program holds itself to.

The architecture is two engines, each strong on one of the two things a language model is asked to do:

• The prediction engine reads the next character. It is the diverse-view count bank, pooled by the calibrated BLEND-NORM geometric mean, growing its capacity with data by grow-then-prune. It combines evidence by reuse: a context key seen before accumulates counts, and many views vote. It reaches held-out bits-per-char near 2.1 on text8, the n-gram level, gradient-free and count-native.
• The abstraction engine builds a generalizing internal code. It is the sparse apical stack: k-WTA with boosting, positive weights, and per-unit precision-gated apical credit (the P0.5 configuration). It reaches transfer abstraction (CCGP) at or above the 0.484 backprop ceiling (about 0.485 to 0.52) with no gradient at all, the genuinely novel result of the program. Its code is high-rank and generalizing, and nearly unique per position.

The two engines do not collapse into one code. Put a count head on the abstract code and it pays a stable about 0.45 bits-per-char tax against the same count head on the raw context (the two-head result). That is the rank/reuse tension: the rank a code needs to abstract is the rank that makes it too unique to count. It is the program's central open problem, and the next direction is the cortex's own answer to it.

The rest of this page walks both engines in detail: the shared random-SDR encoder; the prediction engine (the diverse-view bank, the BLEND-NORM pool, grow-then-prune); the abstraction engine (the sparse apical stack level by level, with the credit wire and the stabilizers); a dedicated section on the dimensions (the knobs the experiments characterized and their sweet spots); the rank/reuse tension; the coupled-laminar next direction; the readout and the generation loop; the attention and working-memory subsystem; and finally what is deliberately not in the architecture.

The two engines, at a glance

One substrate idea, read two ways. Both engines are built from the same parts the program trusts: online counts, a sparse high-dimensional code, a precision signal, and diversity pressure. They diverge on what they keep. The prediction engine keeps a reusable key (the raw n-gram, optionally tagged) so evidence accumulates across positions; the abstraction engine keeps a high-rank code (which units fire) so the geometry generalizes. The same precision signal is read three ways across both: as attention and vote gain, as working-memory eligibility, and as the per-unit credit amplitude. The diagram traces the abstraction engine and the generation loop; the prediction engine is the count bank that feeds the same readout and the same precision read.

1 · Input: the random-SDR encoder (shared)

The model never sees raw characters. It sees a fixed-random sparse distributed code of the recent context, and both engines read it.

The alphabet is 27 symbols: a to z are 0 to 25, space is 26. Each letter is assigned one fixed random sparse code (k active bits out of n, about 2% density) drawn once from a seed and never changed. There is no learning in the encoder; it is a frozen, deterministic dictionary (LetterSDR in lib/sdr_encoder.py).

The context window is the last W = 5 characters. To encode it, each of the five letters' codes is position-tagged (rolled by offset × stride bits, so the same letter in a different slot lands on a different bit set) and all five shifted codes are OR'd into one input SDR (ContextEncoder.encode). Position-tagging is what lets the layer above tell "the a two back" from "the a just now"; OR-ing keeps the input at a fixed size n no matter how wide the window, and makes similar contexts (sharing letters at the same offsets) share input bits, the overlap a pooler needs to map similar contexts to similar codes.

For the abstraction engine the SDR is then taken dense and L2-normalized into one vector of dimension n_in = 1024 (featurize_dense in lib/mlx_learner.py). That normalized dense vector is the input to L1. The prediction engine instead reads raw character spans at chosen offsets directly (its views), the count-native form of the same context.

Why a fixed random code and not a learned embedding: a learned embedding would need a gradient, and the whole point is to build abstraction without one. The random SDR is a high-dimensional, sparse, content-preserving substrate the stack can carve, exactly the role the brain's input projections play in the geometry the program is chasing (the PFC neural-geometry result grounds this: learning shapes the geometry from high-dimensional-random toward low-dimensional rule-selective).

2 · The prediction engine: diverse views, voting, and growth

The prediction engine is the program's best next-character predictor. It is count-native, online, gradient-free, and bounded, and it does what the abstraction engine cannot: it turns more data into lower bits-per-char.

The diverse-view bank

The engine is a bank of count columns. Each column reads a different view of the recent context, (offset, span, feature): a raw character span at some look-back offset and width, plus content cues like the AO-style word-position. Each column is a small conditional count table, NARS f·c-calibrated, predicting the next character from its own vantage. The columns are deliberately diverse: a different view per column. Diversity is what makes voting earn its keep. Same-view voting is worse than a single column (the product-of-experts over-sharpens N copies of one distribution); different views decorrelate and combine, the Monty d_t≠0 "different sensor patch" confirmed empirically. Breadth is a prediction organ. It belongs to the prediction engine, not the abstraction engine, where breadth raises dimensionality without raising abstraction.

The BLEND-NORM pool: a calibrated geometric mean

The columns are combined by BLEND-NORM, a calibrated geometric-mean pool. This is the load-bearing calibration of the engine, and it is the corrected default. The earlier pool was an un-normalized product of experts: it raised the consensus distribution to the power of the summed confidences (an effective exponent about 3.3 over ten active columns), which over-sharpens and inflates the bits. BLEND-NORM normalizes that exponent back to about one, keeping every column at the right temperature. It reads about 1 bit-per-char better than the over-sharpening product pool, and it beats hard winner-take-all (Global Workspace ignition) at every scale: all-or-none access throws away evidence the blend keeps and uses (the vote was too loud). BLEND-NORM also corrected an inflated absolute bpc: experiments built on the un-normalized pool read about one bit high in absolute terms, with their within-experiment comparisons still holding because every arm paid the same tax.

Growth: grow-then-prune breaks the plateau

A fixed bank saturates. Twelve fixed columns improve to about a million characters and then plateau (2.234 to 2.112 to 2.109 bits-per-char, flat from a million on), while a plain n-gram keeps falling because it keeps growing its context table. So the prediction engine's capacity grows with data. The bank adds a longer-context voting column when the held-out gain stalls (the columns have saturated), caps the count, and prunes the least-used at the cap (GrowableBank). On the identical text8 split the growable bank goes 2.234 to 2.004 to 1.875, growing from about 12 to 24 columns, where the fixed bank flattens at 2.109; it grows the right thing (progressively longer-context views) and holds the budget at the cap (growth breaks the plateau). So the prediction engine restores the n-gram-like scaling slope by doing what the n-gram does, under a memory budget.

The honest place of the prediction engine

The number is the n-gram level, and that is honest. The growable bank reaches about the n-gram floor (1.875 against 1.829), not past it; it is still about 0.85 bits-per-char behind text8 SOTA near 1.0; and prediction is not the program's headline axis. The bet is what the n-gram and a tiny transformer lack: the abstraction the second engine carries above the gradient ceiling, the online-bounded-gradient-free regime, and the attention and generation machinery. The prediction engine is a mile from a strong transformer, not a hundred light-years: in the right universe, far from the frontier, and it keeps learning from data instead of saturating.

3 · The abstraction engine: the sparse apical stack

This is the heart of the program, and the part the experiments worked hardest to settle. It builds an abstract, generalizing internal code with no gradient, the result that does not exist elsewhere in the gradient-free literature. In code it is the P0.5 +BOOST+POS configuration of KWTADiversityStack, the stabilized depth-three apical stack. The configuration is depth-3, k-WTA sparse, boosting plus positive weights, per-unit precision-gated apical credit, with actnorm plus gate-clip at act_gain ≈ 0.2. Every clause of that sentence was bought by an experiment. Here is the stack, piece by piece, with the reason each piece exists.

The shape: a depth-3 sparse stack with a next-char readout

The forward shape is a straight climb. The input SDR (n_in = 1024) feeds L1; L1 feeds L2; L2 feeds L3; and L3 feeds the readout, a linear softmax over the 27 next characters. Three hidden levels, each a sparse code: instead of a dense ReLU layer, each level takes a k-WTA code (the top k units fire, the rest are silent), so the activity count is bounded and the code is a sparse distributed representation. The forward pass is ordinary up to the k-WTA; what is not ordinary is how the weights learn, because there is no backprop anywhere.

Why depth 3, and not more. Depth is what buys altitude: each level out-abstracts the one below, and abstraction (CCGP) climbs L1 → L2 → L3. But it climbs to three and then saturates: depth-4 adds dimensionality without adding abstraction, and the deep levels collapse. That ceiling was established by scaling the stack (three swings after the keystone) and confirmed when the capstone meant to push past it ran and failed (the architecture is locked). Three is the settled depth.

The apical credit wire: the abstraction itself

Between every adjacent pair of levels there is a second wire running the other way: a per-unit, top-down apical credit signal. This is the mechanism that makes the stack abstract, and it is the single most important idea in the engine.

The readout's error is dz = p − onehot(target). That error is routed back into the hidden levels not through the transpose of the forward weights (that would be backprop, which needs weight transport, a neuron reading its own outgoing synapses, which biology cannot do), but through fixed random feedback matrices: B_top from the readout to the top hidden level, and an apical wire Bap[ℓ] (an H × H fixed random matrix) from each level to the one below it. This is feedback alignment: the credit each unit receives is per-unit and local, carried down a real top-down pathway, with no weight transport. In the sparse stack the credit is masked to the k winners, so only the units that fired are taught. The forward weight then takes a local delta step against that per-unit credit.

The credit flows level by level, top to bottom:

  dz   = p − onehot(target)
  Wo  -= fa_lr · outer(h_L, dz)              # readout: its own local delta
  d_L  = (B_top · dz) · [winner_L]           # top level: error → top hidden via fixed B_top, masked to winners
  W_L -= fa_lr · outer(h_{L-1}, d_L)         # train the top forward weight (ungated)
  for ℓ = L-1 … 1:                           # carry the credit DOWN, level by level (apical)
      d_ℓ  = (Bap[ℓ] · d_{ℓ+1}) · [winner_ℓ] # per-unit credit from the level above, masked to winners
      d_ℓ  = normalize(d_ℓ)                  # RMS-normalize the credit (gate-clip)
      W_ℓ -= (ap_lr · g) · outer(in_ℓ, d_ℓ)  # GATED by the precision burst g

The same top-down wire that carries this credit is the wire that would prime the level below during a forward sweep: priming and credit are one mechanism. This is the architectural keystone, and the reason the engine exists in this form.

Why per-unit, and why this pathway. The program walked the abstraction question all the way down. First it proved the missing ingredient was credit assignment, not wiring: holding one architecture fixed and swapping only the update rule, a gradient builds an abstract space while a no-gradient Hebbian rule compresses hard but loses abstraction (the abstraction wall was credit assignment). Then it sharpened "credit assignment" to per-unit credit: a broadcast neuromodulatory scalar fails (one number everywhere can only scale an unsupervised step), while a per-unit feedback-alignment signal clears the wall (a local signal clears the wall). Then it answered on what pathway, how biologically: carry that per-unit credit down a real apical wire, and the result is the first stack whose abstraction climbs with depth (the apical fusion). The apical wire is not decoration; it is the abstraction.

The precision signal: the third factor that gates the credit

The credit does not flow at a constant rate. Its amplitude is gated by a precision/surprise burst, the third factor. On each step the model reads its own surprise on the true character, s = −log₂ p(target), tracks a running mean of it, and forms a bounded burst g = g_floor + clip(s − running_mean, 0, burst_max). The apical (below-top) updates are scaled by g; the top forward weight and the readout are left ungated, so the gate's effect is isolated to the apical wires.

This is the "one precision read three ways" principle: the same precision burst is (1) the attention/vote gain that sharpens confident columns in the prediction engine, (2) the working-memory eligibility, and (3) the per-unit credit amplitude here. It says when to learn hard (on the surprising characters) while the per-unit feedback says how each unit should change. Precision-gating is load-bearing: the gated apical fusion (transfer CCGP 0.501 on the original two-level stack) decisively beat the ungated wire (0.355).

The diversity machinery: boosting and positive weights

A bare sparse code with per-unit credit abstracts and then collapses: the dense credit pathway drives the same few units to win for every input, so the high-rank sparse code degenerates to a handful of winner-sets (the winner-collapse). The cure is two pieces of diversity machinery, and it takes both (the empty cell, filled).

• Boosting / duty-cycle homeostasis is the primary anti-collapse piece. Each unit tracks how often it wins, and a unit that has been winning too much is down-weighted so others get a turn. This keeps the winners diverse across stimuli, so the sparse code stays high-rank instead of collapsing to a few codes. It is the homeostatic spread that the whole abstraction depends on.
• Positive weights and no bias (the SDM / Bricken support) are what let boosting work. With signed weights the credit can drive a few units to always win and homeostasis alone cannot overcome it; clamp the forward weights non-negative and remove the bias, and then boosting keeps the winners spread. Boosting alone still collapses; the minimal sufficient pair is both, together.

With both, the depth-3 sparse stack holds deep transfer abstraction at the 0.484 ceiling to the last checkpoint on both seeds, with more than two hundred distinct winner-codes (against the dense reference's about forty) and the lowest bits-per-char of any arm. A sign-only permanence credit (the textbook HTM step) breaks the code, so the credit stays graded, with diversity machinery around it, not replaced by a sign.

The stabilizers: actnorm and gate-clip

Feedback alignment, stacked with a precision gate, diverges at depth without help: the forward activations grow unbounded, the credit is an outer product of those activations so it blows up too, and the surprise gate amplifies the runaway. Stability, not depth, not data, was the wall. Two pieces, applied to every level, solve it.

• actnorm, forward-RMS normalization. After each level, the hidden vector is rescaled to a fixed target RMS, act_gain. This is homeostatic gain control: the forward can no longer blow up, so the credit stays bounded too. actnorm is necessary and sufficient for stability: it holds the stack stable at every depth, rate, and data size where the baseline, a gate-clip alone, and a dream-replay cycle alone all diverge into the teens of bits-per-char (the architecture is locked).
• gate-clip, bounded precision burst plus RMS-normalized credit. Stability is not abstraction: plain actnorm stays stable but collapses the deep code to one dimension. Gate-clip keeps the code healthy: it tightens the gate's ceiling and RMS-normalizes the apical credit d to a fixed scale before the gated step, so no single level's update can blow up regardless of the gain.

Both are required: actnorm for stability, gate-clip for an abstraction that is real rather than collapsed.

The headline result

Put together, the abstraction engine produces what every prior gradient-free stack in the program failed to produce: rising-altitude abstraction. CCGP climbs L1 → L2 → L3, the transfer score reaches about 0.485 to 0.52, at or above the 0.484 single-layer backprop ceiling, and it is stable and seed-robust. The multi-view unified form reaches 0.520 at higher dimensionality with no collapse, the highest the program has reached. This is a biologically-shaped, online, local, bounded learner that builds an abstract code without a gradient, the named prize of the abstraction line, in hand. (The 0.484 mark is the single-layer online backprop reference on this harness; a multi-view backprop is the fair ceiling to recheck before any claim against deep backprop.)

4 · The dimensions

The two engines are not a fixed point; they are a set of knobs the experiments characterized, each with a sweet spot and a job. This section is the standing map of those dimensions: the knob, its sweet spot, what it buys, and the experiment that bought it.

A few of these read together. Depth belongs to the abstraction engine; breadth belongs to the prediction engine; and the breadth-times-stability combine is non-additive, because breadth preserves variance across columns while normalization destroys it across stimuli within a column. Learning rate is the just-published gentle learning compounds result: 0.001 reaches a held-out 3.637 against the aggressive 3.762, 0.125 bits better, and the optimum is a U; the rate that proved too hot is the apical stack's global rate, so the implication reaches past re-reading. Exposures: re-reading lowers held-out bpc from 3.863 to 3.762 over five passes at the locked rate, and to 3.637 at rate 0.001 over ten passes; most of the gain lands on the first re-read and it plateaus by the third; the count voting bank is re-reading-invariant (a count table is complete after one pass). Boosting and positive weights are the diversity machinery, and per-unit apical credit is the abstraction wire; together they are the abstraction engine's core. Growth and BLEND-NORM are the prediction engine's two characterized knobs.

The rank/reuse tension

The two engines do not fuse, and the reason is structural. This is the program's central open problem, named precisely.

Take the abstract code that abstracts at the ceiling and put a count-native prediction head on it, against the same count machinery on the raw context, on the same frozen substrate. The count head extracts real signal from the abstract code: it reads 2.578 bits-per-char at one hundred thousand characters, 0.8 to 1.0 below the stack's own softmax readout (3.548), so one substrate can feed both heads. But it stays about 0.45 bits-per-char behind raw context (2.141), and the gap holds stable across 5.5 times the data (+0.50 at eighteen thousand, +0.44 at one hundred thousand), while the same code's transfer CCGP is 0.499, above the reference (the rank-reuse tension). The same code abstracts well and predicts worse.

The mechanism is a tension in the representation itself. A table-fill diagnostic shows the abstract code is nearly unique per position: even at seventy thousand training positions a full winner-set key is seen only three to six times. The high rank that makes the code abstract (hundreds of distinct winner-sets, mixed selectivity) is exactly what makes it almost never repeat, and a count table predicts by reuse, so it backs off to coarser subsets and predicts bluntly. Raw context repeats far more, so its count table is dense and sharp. So one code cannot be both abstract (high-rank, unique per position) and count-reusable (must repeat): the rank a representation needs to abstract is the rank that makes it unique, and a unique code cannot be counted. That is why the program keeps drawing two engines, and why one shared code provably cannot reach the low-bpc-and-high-CCGP corner at once.

The next direction: coupled-laminar layers

If one code cannot be both, keep both codes and couple them, the way the cortex does. This is the program's bet to resolve the tension, and it is sourced from the canonical cortical microcircuit.

The literature returns a clear verdict: the tension is not a bug to optimize away, it is why the cortex uses different layers. Bastos 2012 (canonical microcircuits for predictive coding) maps the two jobs onto layers and bands: superficial L2/3 carries feedforward prediction-error (gamma); deep L5/6 carries feedback prediction (beta/alpha), distinct populations with distinct jobs. That is the external license to stop merging the two codes, and it names the coupling in both directions. Larkum 2013 (a cellular mechanism for cortical associations) gives the cellular warrant: one L5 pyramidal cell binds the two streams, basal dendrites = bottom-up/feedforward, apical tuft = top-down/feedback, electrotonically segregated, their coincidence within about 20 to 30 ms gating plasticity. The abstraction engine's per-unit precision-gated apical credit is already a Larkum cell (per-unit apical credit, gated by the precision burst, masked to the winners); the program simply never used the apical pathway to carry a second engine's signal. The cell binds two streams without merging them, because they enter different compartments.

So the design keeps both codes and shares information, not representation, across two named seams:

• Feedback (apical), abstraction → prediction. The abstraction engine's settled code, quantized to a coarse abstract cluster-id so it repeats (the full high-rank SDR is too unique to count, which is exactly the two-head tax), is appended to the prediction engine's context key: a context-conditioned count column. Coarse enough to recur, informative enough to help.
• Feedforward (basal), prediction → abstraction. The prediction engine's pooled code is concatenated into the abstraction stack's basal input as a disjoint block, giving the abstraction stack the predictive surface detail it currently throws away.

The cheap first slice has run: a count-only test of the feedback seam, asking whether a coarse abstract tag informs prediction (a tagged count column beats the raw n-gram) without simply fragmenting the counts (a shuffled-tag control isolates the tag's real information from the key-splitting cost). The verdict is an honest negative for the next character (abstraction cannot cheapen prediction): the short-horizon abstraction tag carries real information (it beats its shuffle by up to 0.385 bits-per-char) but is redundant with the order-six n-gram, so a soft coupling reaches break-even and no better (+0.001 at a million characters), and a long-horizon tag carries almost no next-character information at all and hurts, even at the least-local word-start positions. Information is not usable prediction gain. This matches the sources' own honest limit, that this is a predictive-coding circuit and predictive coding is a local gradient, so it does not promise to climb past the 0.484 ceiling. But it also reframes the seam: the next character is an overwhelmingly local game counting already wins, so the cross-feed is the wrong ruler there. The forward arrow is already built (the abstraction engine's per-unit credit is the prediction error, so prediction already teaches abstraction), and the cross-feed must be measured where abstraction lives, the next word and generation coherence, not the next character. The soft-coupling mechanism is validated and ready for that scale, and that is the path on to the larger architecture: generation and production, and learning by example.

5 · Readout: comprehension and calibrated confidence

The top hidden level feeds a linear softmax over the 27 characters: that is the next-character prediction, the comprehension output. The readout learns by its own local delta against dz (no transport).

Alongside the prediction the model exposes a calibrated confidence, the precision read. A count split into hits and misses gives a NARS truth value (f, c) whose product f·c is calibrated essentially for free (how sure is a count?), and this is the same precision/surprise signal that gates the apical credit. So the readout emits two things: what comes next, and how sure the model is, and that confidence is not a separate module, it is the third factor read off the prediction itself.

6 · The generation loop: SF1, the internal half (nascent)

With both engines characterized, the program's open value moved to generation: producing text, not just modelling it. Generation has two halves; the internal half landed its first positive in experiments/exp_sf1_selffeedback/run.py, the birdsong loop. A songbird learns its song from self-hearing alone, with no tutor present: it babbles, hears itself, and corrects toward the song it means to sing. SF1 builds that loop for text. It is online, gradient-free, bounded, and needs no environment.

One iteration of the loop:

1. Set the target. An intended meaning m* (a real held-out word the comprehension pathway already represents) is fixed, independently of the producer (the anti-trivial-fixed-point guard). The meaning is stored as a salience-weighted "heard fingerprint," not a raw string, so the producer cannot cheat by string-matching; it must produce something that sounds like the meaning to comprehension.
2. Babble. The producer samples candidate utterances over the committed-chunk emission vocabulary (ChunkLexicon), whole committed units, not characters, biased by its learned per-meaning chunk counts plus an explore term that anneals down as competence rises.
3. Re-comprehend. The same comprehension pathway reads each candidate utterance and recovers a posterior m̂ over meanings, the forward model pointed inward. This pass is read-only: it never writes the comprehension counts.
4. Self-error. e = 1 − m̂[m*], one minus the mass the re-comprehension puts on the intended meaning. The NARS f·c confidence on the utterance is logged as a cheap second view.
5. Correct, win-stay / lose-shift. A small error reinforces the chunks that were used; a large error weakens them. The self-error is consumed as a scalar reward by count reinforcement: no gradient is routed back through the comprehension decoder.

The result. Against a chance of 0.042 (one of 24 meanings), the full loop recovers an intended meaning 0.271 of the time: about 6.5× chance and 8× its own open-loop and deafened ablations. All three guards fire: scramble the target and it collapses to exactly chance; let the self-output contaminate comprehension and an independent frozen judge catches the private-code drift. So internal self-feedback teaches production with no listener (the generation frontier, opened). A separate learned inverse then carried this to real words: pointed at the top 300 DailyDialog words it produces 113 of 300 real words and recovers the intended one 0.317 of the time against the reader run backwards at 45 and 0.090, real English (lady to ready, agreed to good), and it holds a live conversation with a real Haiku partner, word-salad with real glimmers of relevance.

The honest scope. Absolute recovery is modest, a first cut on a coarse meaning code. By design the loop only ever agrees with itself: it supplies fluency, not a shared convention. Convention needs the external half: a referential game scored on whether a real listener recovers the referent, the anchor against private-code drift, and the falsifiable next move. The loop is the architecture's nascent speaking organ; the two engines are its settled understanding organs.

7 · Attention and working memory: a validated subsystem

A second altitude rides on top of the prediction columns: a learned attention system that decides where each column looks and what the level above holds while it reads. The whole stack is online, gradient-free, and bounded, and at the payoff scale it beats hand-set fixed views. It came together over three experiments, each adding the one piece the last one named, and a fourth that bounded it.

The encoder reads a fixed window a fixed distance back. The attention subsystem makes that distance a choice. Each column reads a view, a window a chosen distance back over a fixed span, and the attention is the policy that picks the distance.

• Each column learns where to look. The look-back offset is an action, chosen fresh every step and rewarded by how well the column then predicts. A count-based, gradient-free bandit over a menu of offsets learns the choice, the menu holding a content-relative word-jump. The wire that makes it work is lateral inhibition on the action space: a column is docked for predicting well where the other columns already look, the same diversity pressure that clears representational collapse, carried from the code to the scan. Without it every column collapses onto the single most recent character; with it the columns spread (learning where to look).
• A higher level guides the scan by context. A working memory one level up steers it: a leaky char-distribution over the region recently read plus the live word-phase. Each column's offset bandit is keyed on it, so the column learns in context X, look at offset Y. At the right granularity this beats the hand-set baseline on prediction (3.022 against 3.101) and lifts abstraction at every granularity (where you look depends on what you understand).
• A gate decides when to update the held theme. A basal-ganglia-style Go/NoGo gate replaces the fixed leak. It fires on a model-update signal, how far the column pool's own prediction moved step to step (the shift carves events where surprise carves words), with an adaptive tail threshold, learned online and gradient-free with no reward. It beats the fixed leak (3.069 against 3.083) and both degenerate bounds, holds the theme about eight characters, and fires at a word boundary three times in four (a gate for working memory).

The arc closed with a bound: two levels of guidance with a self-calibrating gate is the sweet spot, a third nested level costs more bucket-splitting than its longer-range guidance gains (two levels is enough).

The honest scope. This subsystem runs on the diverse-views count columns, the prediction engine's substrate, distinct from the abstraction engine's apical stack. Crucially, the attention representation is a prediction and working-memory win, not an abstraction one: fed verbatim as the input to the per-unit-credit readout and measured on the gradient-comparable ruler, it abstracts no better than a plain context window, so its apparent CCGP lift was expressive dimensionality, not abstraction (attention is prediction, not abstraction). It is validated on its own substrate and scale, the prediction side of the two-engine split.

8 · What is deliberately not in the architecture

The experiments factored the architecture cleanly, and several validated mechanisms are deliberately kept out of the abstraction engine's core single column, each belonging to a different job. Stating this plainly is part of the spec: the honesty bar is as load-bearing as the wins.

• Multi-attention / breadth is out of the abstraction core. Different attended views per column decisively help prediction and keep the code high-dimensional, but they leave abstraction flat. So prediction and abstraction factor at the engine level: prediction from diverse precision-weighted views, abstraction from per-unit apical credit. The multi-column machinery is the prediction engine; the abstraction engine's core is the single-column stack (three swings after the keystone).
• The breadth × stability combine ran and failed; the levers are not additive. It was natural to expect breadth to rescue actnorm's deep collapse. The "A1" capstone tested exactly that and the prediction was false: the combine stays stable but collapses to dimensionality 1.17, because the two levers act on different axes of variance (the architecture is locked).
• One shared code that both predicts and abstracts is out, for now. The two-head experiment measured the cost directly: a count head on the abstract code pays a stable ~0.45 bpc tax. The rank/reuse tension says one code cannot be both, so the architecture keeps two engines rather than one shared code, and the coupled-laminar next direction shares information across them instead. The lever the diagnostic points at (a coarser, less-sparse code at a sparsity-versus-reuse sweet spot) is recorded as untested.
• Deeper than 3 saturates. Depth-4 adds no abstraction and the deep levels collapse even when stable. Pushing past three needs a genuine design change of uncertain payoff, since we are already at the ceiling. It is an open residual, not a closed door.
• The drift on a long fresh stream is mitigated, not solved, so the mitigation stays out of the locked core. The locked config holds above the ceiling on a bounded corpus, but on a long fresh non-stationary stream it drifts. Two mechanisms push that back, lateral divisive inhibition between same-level columns and K-line replay reinstated at an interior altitude, and they compose, but the ten-million confirmation came back partial: the deep code stays alive where the bare config collapses, yet abstraction slips below the ceiling at scale. So the drift is mitigated, not cured, and the two wires are kept out of the locked core until a tuned, multi-seed run holds the protection above the ceiling at scale. An open continual-learning residual.
• A label-attractor shortcut to abstraction was ruled out. A self-supervised label-anchored prototype pull compresses dimensionality harder than anything in the line (PR 5.8) but scores below the raw baseline on transfer: compression is not abstraction. The gradient-free route to abstraction is per-unit credit, not a label attractor (three swings after the keystone).
• The count-native mode-selector is a closed negative. A bank of self-calibrating critics with a selector over learning modes loses to the trivial fixed policies, because the one selective signal (surprise) gates the harmful mode and the helpful mode (replay) wants a dose, not a tail (a selector that cannot win).

Version history / Changelog

The architecture's evolution, newest first. Each entry is a published version; prior full versions are kept as snapshots (/architecture-v<N>/) so this page always shows the latest while nothing is lost. When an experiment shifts the architecture, this spec is re-versioned: the current full spec is copied to a /architecture-v<N>/ snapshot, this page is rewritten as the new full version, and an entry is appended here noting what changed and which experiment drove it.

• v2 · 2026-06-28 · the two engines. The organizing reframe: the architecture is now two engines, a prediction engine (the diverse-view count bank, the BLEND-NORM calibrated geometric-mean pool, and grow-then-prune growth, near 2.1 bpc at the n-gram level) and an abstraction engine (the sparse apical stack at or above the 0.484 backprop ceiling, gradient-free). The prediction engine, BLEND-NORM, and grow-then-prune are now first-class (v1 treated multi-attention only as a prediction organ kept out of the core; v2 makes the prediction side an engine of its own). The dimensions are characterized in a dedicated table, including the learning-rate sweet spot (0.001, a U-shaped optimum, the locked rate too hot). The rank/reuse tension is named as the central open problem: one code cannot be both high-rank-abstract and count-reusable, the ~0.45 bpc two-head tax measured directly. And the coupled-laminar next direction is set out: keep both codes, couple them as Bastos 2012 and Larkum 2013 describe (the abstraction engine's apical credit is already a Larkum cell), sharing information not representation across two named seams, with the cheap count-only first slice running. Experiments that drove it: the section 6ae-6am arc, the two-engines lineage (BLEND-NORM 6ae, grow-then-prune 6ai, the P0.5 abstraction 6ah, the unify trade-off 6aj, the two-head tax 6al, multi-exposure 6ak, the learning-rate sweep 6am). This folds the two 2026-06-28 v1 notes (the partial drift-mitigation and the attention/working-memory subsystem) into the v1-to-v2 story: both are recorded here, the attention subsystem now framed as the prediction side of the two-engine split, the drift mitigation as a kept-out continual-learning residual. The full v1 spec is kept at /architecture-v1/.
• v1 · 2026-06-27 · initial architecture spec. The first canonical spec, written after the abstraction core was locked above the backprop ceiling (experiments DC → DE → DH → DI/DJ/DK/A1) and the generation loop opened with its first positive (SF1). Covered the random-SDR encoder, the depth-3 per-unit precision-gated apical-credit core with actnorm plus gate-clip at act_gain ≈ 0.2, the calibrated readout, and the internal self-feedback generation loop, and stated what is deliberately kept out of the single-column core (multi-attention/breadth as a prediction organ, the non-additive breadth × stability combine, depth past 3, the label-attractor shortcut). Two same-day notes were appended without a version bump: the ten-million drift-mitigation confirmation came back partial (the lateral-inhibition plus interior-replay pair keeps the deep code alive but abstraction slips below the ceiling at scale, so it stays out of the locked core), and the attention and working-memory subsystem was added as a validated subsystem (experiments AT1 → AT2 → AT3, columns learning where to look with lateral inhibition, a higher level guiding the scan by context, and a basal-ganglia-style gated working memory). Both notes are folded into the v2 story above. The full v1 spec is kept at /architecture-v1/.