Abstraction cannot cheapen prediction

Theory update · 2026-06-28 · the feedback seam · experiment coupled-laminar, slice 1

The last round found a tension: one shared code cannot be both abstract (high rank, nearly unique per position) and count-reusable (it has to repeat), so a count head on the abstract code pays about half a bit. The cortex's own answer is not to merge the two codes but to keep both and let them share information through a coarse, repeatable tag, the feedback arrow from the abstraction engine to the prediction engine. This builds that seam, count-only, the cheapest decisive test, and the result is an honest negative with a clean diagnosis. The tag carries real information and still buys no prediction. A short-horizon tag, the abstraction stack's own deep code, beats a shuffled version of itself by up to 0.385 bits-per-char, so it is genuinely informative, but it is redundant with the order-six n-gram (it re-encodes the same six characters), so a soft coupling that never displaces a well-counted raw cell reaches break-even and no better: at a million characters the net is +0.001, exactly the baseline, never below it. A long-horizon tag carries almost no information at all. A recency-weighted gist over the characters outside the n-gram window beats its shuffle by about 0.013 bits, no better than a random tag, and conditioning on it only hurts (+0.06 to +0.10). And it does not help even where local context is weakest: at word-start positions, the least locally determined in the text, the long-range tag still costs +0.05 to +0.11, with no scale dependence. The unified finding is sharp. Next-character prediction is an overwhelmingly local game that counting already wins, and information is not usable prediction gain: where there is information it is redundant, and where there is new context there is no information. So the conclusion is not that the coupling fails. It is that we were holding the wrong ruler. The mutual loop is already half-built, because the abstraction stack's per-unit credit is the prediction error, so prediction already teaches abstraction; the only missing arrow is the one this result shows is null for the next character, and the place to measure it is the next word and the coherence of generated text, not the next character.

The question

The program has split into two engines that do not fuse into one code. A count-native voting bank predicts, low bits-per-char, because count tables combine: a context that recurs accumulates evidence and the prediction sharpens. A sparse apical stack abstracts, transfer CCGP at the backprop ceiling, because per-unit credit carves a high-rank, factorized code. The rank-reuse tension measured why one code cannot do both: the high rank that earns the abstraction is exactly what makes the code nearly unique per position, so it almost never repeats, and a count table predicts by reuse. Counting on the abstract code paid a stable half-bit tax against counting on the raw context, and the gap held across more than five times the data.

That tax is a fact about merging the two codes into one shared key. The cortex does not merge them. The coupled-laminar reading of the canonical microcircuit (Bastos 2012, Larkum 2013) says the brain keeps a feedforward error stream in the superficial layers and a feedback prediction stream in the deep layers, in different layers, and couples them. The lesson is the license to stop merging the codes: keep both, and have them share information, not representation. The abstraction engine's settled code, quantized down to a coarse, repeatable cluster-id so it actually recurs, becomes a tag on the prediction engine's reusable raw-context count key. The full code is too unique to count on, which is the tax itself; a coarse quantization of it repeats often enough to be a usable context tag on top of the raw key.

So the question is the feedback arrow, abstraction to prediction, asked at its cheapest. Does conditioning the count predictor on a coarse abstraction tag buy a prediction gain the raw n-gram does not already have?

The bet, and the arms

The bet, stated to be falsifiable. A coarse abstraction tag, conditioned on the reusable raw code rather than replacing it, nets a prediction gain. A win is two things at once: a net gain (the tagged predictor beats the raw n-gram) and the tag carrying information (the real tag beats a shuffled version of itself). The first is the prize; the second is the control that says any gain came from the tag's alignment to context, not from merely splitting the counts into more buckets.

Four readings, all count-native and gradient-free, the same add-alpha and backoff, scored on the identical text8 split and bits-per-char loop the head-to-head uses, so the numbers sit directly beside the rank-reuse tax. The arms differ in one thing only, the key.

• A, raw-alone. The order-six backoff n-gram on the raw character context. The strong local baseline, the prediction engine on its own.
• B, hard conjunction. The tag folded into the count key, so each cell is (raw n-gram, coarse cluster-id). A two-ladder backoff falls from the tagged orders to the raw orders to the unigram floor, so B can never fall far below A.
• E, soft coupling. Instead of conjoining the tag into the key, interpolate the tagged and raw conditionals at each order, mixed by how well-counted the tagged cell is. A well-supported tagged cell is used; a sparse one falls back to the raw cell. So the tag can only add value where it has evidence and never displaces a well-counted raw cell. This is the fix for the fragmentation that the hard conjunction causes.
• C, the shuffled-tag control. Arm B with the training tag stream permuted, the same tag multiset with its alignment to context destroyed (evaluation still uses the real tags). This isolates the tag's real information from the cost of splitting the counts. B minus C is the information; E minus A is the soft net against raw.

The tag granularity, how many of the deep code's winners go into the cluster-id, is the rank-reuse dial: too fine and it is unique again, too coarse and it carries nothing. We sweep it, at two hundred thousand and a million characters.

What happened

The headline is the contrast between two tag types, and it is the whole finding in one picture.

Three slices fill it in. All bits-per-char on held-out text8, lower is better, positive deltas are worse.

Slice one, a short-horizon tag. The tag is the abstraction stack's own deep code, the top-g winners of the P0.5 sparse apical stack, the same code whose transfer CCGP sits at the backprop ceiling.

The tag carries lots of information. It beats its own shuffle by up to 0.385 bits-per-char, and the margin grows with g, because more winners means more alignment for the shuffle to destroy. But the information is redundant with the order-six n-gram: the deep code reads the same handful of recent characters the n-gram already counts. So the hard conjunction fragments the counts and B sits above A everywhere, though the penalty halves from two hundred thousand to a million as the tagged cells fill. The soft coupling removes about half that fragmentation and trends to break-even, at g=1 and a million characters the net is +0.001, exactly the baseline, but it never goes below A. Real information, no usable gain.

Slice one-b, a long-horizon tag. If the short tag failed because it shares the n-gram's horizon, then a tag outside that horizon is the fair test. This one is an order-free, recency-weighted letter gist over the characters from sixty-four back to seven back, strictly outside the n-gram's last six, sign-hashed to g bits, a poor-man's long-range topic id with no neural stack at all.

Here B minus C is essentially zero, about 0.013 bits at a million characters and not growing with g. The long-range gist carries almost no information about the next character, no better than a random tag of the same cardinality, and conditioning on it only fragments the counts, so it hurts. A horizon outside the window does not rescue the seam; it has no next-character signal to give.

Slice one-c, the word-start cut. Maybe the long-range tag helps only where local context is weakest. So restrict the score to word-start positions, the characters whose target is the first letter of a word (the previous character is a space), the least locally determined and most topic-dependent positions in the text.

Word-start prediction is hard (the baseline sits near the no-skill ceiling of 4.755), exactly where a long-range prior should earn its keep. It does not. The long-range tag costs at every granularity and scale, and shows no scale dependence, no sign of turning helpful as the corpus grows. There is no character-level slice where the abstraction tag pays.

What this means

The two failures are different, and together they close the door on character bits-per-char. Where the tag carries information (slice one), the information is redundant with the n-gram, so a soft coupling reaches break-even and no better. Where the tag brings new context (slice one-b and one-c), it carries almost no next-character signal, so it cannot help even at the least-local positions. Information is not usable prediction gain.

This is the same shape the program keeps finding. Attention is prediction, not abstraction showed a richer input did not abstract better on the gradient-comparable ruler; here a more abstract code does not predict better, the same separation seen from the other side. One stack, two halves showed the vote does not import prediction with neural columns. Abstraction is prediction-orthogonal, and the next character is the place where it shows clearest, because the next character is an overwhelmingly local game that counting already wins.

So the honest reading is not that the coupling fails. It is that we were holding the wrong ruler. Look again at the microcircuit's mutual loop, error up from the superficial layers, prediction down from the deep ones. Half of it is already built in our model. The abstraction stack's per-unit apical credit is the prediction error: high surprise is what marks where a concept boundary should form, the program's original bet, the surprise-carves-structure signal that runs back to the first boundary experiments. So prediction already teaches abstraction. The only missing arrow is the other one, abstraction biasing prediction, and that is exactly the arrow this round shows is null for the next character.

The conclusion follows. Keep both engines good at their own job. Feed them from a shared substrate. And measure the cross-feed at the scale where abstraction actually lives, the next word, the coherence of a generated passage, not the next character, because the next character is local and counting owns it. The soft-coupling mechanism, the support-weighted interpolation that never displaces a well-counted raw cell, is validated and ready for that scale. What it needs is a target where long-range structure moves the metric.

The feedback seam, abstraction to prediction, is null for the next character. A short-horizon abstraction tag carries real information (it beats its shuffle by up to 0.385 bits-per-char) but is redundant with the n-gram, so a soft coupling reaches break-even and no better (+0.001 at a million characters, never below the baseline). A long-horizon tag carries almost no next-character information (about 0.013 bits over its shuffle, no better than random) and hurts, even at the least-local word-start positions, with no scale dependence. Information is not usable prediction gain: where there is information it is redundant, where there is new context there is no information. The reframe: the mutual loop is half-built (per-unit credit already is the prediction error, so prediction already teaches abstraction), and the missing arrow is the one shown null here for char bits, so measure the cross-feed at the word and generation scale, not the next character.

The honest caveats

• It is the theme gist and the short-horizon code that were tested. A sharper long-range signal, the previous word, a learned long-range code, a topic or event tag at the right altitude, is untested, and the unified finding does not rule out one of those carrying both new context and next-character information. The negative is for the tags built here.
• Character bits-per-char only. The whole negative is on next-character prediction. The reframe's claim is precisely that the seam should be measured at the word, meaning, and generation scale, which this round does not reach.
• The substrate is the frozen abstraction hero. The deep code is byte-identical to the one whose transfer CCGP is at or above the 0.484 ceiling, so the tag is a faithful read of the abstraction engine, not a fresh fit. Nothing was tuned to the prediction task.
• Two hundred thousand and a million characters, one seed. The deltas are stable across the five times data, and the slope is the read: slice one's fragmentation halves with scale and trends to break-even, while slice one-b's null does not improve. No evaluation leak: every tag is computed from the causal prefix, and the next character is only the label.

Lineage

Grew from the rank-reuse tension, which showed one shared code cannot be both abstract and count-reusable, and named the cortex's own answer: keep both codes and share information, not representation. This builds the first, cheapest leg of that coupled-laminar plan, the feedback arrow, count-only. It reuses the frozen P0.5 abstraction substrate from the empty cell, filled and the same count machinery the head-to-head runs, so the arms differ only in the key. And it carries forward attention is prediction, not abstraction, the same prediction-versus-abstraction separation seen from the other side.

Led to the reframe that ends this round: the coupling's value is not in character bits-per-char, because the next character is local and counting wins it, so the cross-feed must be measured where abstraction lives, the next word and generation coherence. And to a sharper standing read of the two engines: the feedback arrow (abstraction to prediction) is null at the character level, while the forward arrow (prediction to abstraction) is already built, since the per-unit credit that shapes the abstract code is the prediction error itself.

Thread: abstraction, and online learning without a global gradient. An honest negative that confirms one engine can carry information to the other (the tag beats its shuffle) while showing that information buys no character prediction (redundant where present, absent where new), and names the wrong ruler: measure the coupling at the scale of meaning, not the next character.