What it learns, and what it can say back

2026-06-27 · a wall, read two ways · the coverage baseline and experiment M2 (Minsky)

A standing question, asked plainly: of the words it reads, how many does the model actually learn, and at what altitude? And once it has learned them, can it say any of them back? Two instruments answer. A coverage probe on a controlled vocabulary: the model learns a stable, decodable code for most words at the first level, almost none higher up, and the deep code collapses to a handful of distinct codes. A playback probe on a trained reader: it recognizes hundreds of words and reproduces a few dozen, and the codes it learned play back as generic syllable stubs. It is a strong recognizer and a weak producer. Both readings land on the same wall.

The question

We have measured abstraction with dimensionality and a transfer score for months. Those numbers gesture at what the levels hold without ever naming it. So this asks the concrete version twice. First: build a controlled vocabulary of known words, stream it through the model, and count exactly how many of those words the model learns a stable, decodable code for, at each level, and which it learns versus misses. Second: take a reader trained on real text and ask the other half of the question. It clearly recognizes words. Can it produce them? When you prompt it, do the concepts it learned come back out as the words that went in?

The two are the same question from opposite sides. Coverage asks how much gets in and how high it climbs. Playback asks how much comes back out. A mind that reads well and speaks poorly would show a thin top in the first and a gap in the second. That is what we found.

What we tried

For coverage, a controlled corpus with a ground-truth answer key: five hundred known words, each appearing many times. We stream it through the locked architecture one character at a time, and at the last character of each word occurrence we capture the per-level code, the position where the model has seen the most of the word. The readout is the same nearest-prototype decoder the abstraction probe already uses, no new trained machinery. A word counts as learned at a level if its code decodes back to the right word well above chance. Coverage is the fraction of eligible words learned, per level. Alongside it we count the distinct codes the deep level produces over the stream, the same collapse diagnostic the rest of the work uses.

For playback, the same trained reader and the same decoder, on the comprehension-and-production gap. Comprehension is the coverage read: how many words the decoder recognizes. Production is free-run generation: seed the trained next-character head, sample many characters, cut on spaces, and count how many real vocabulary words come out. And a direct readout of what each learned concept is: prompt the reader with a word's opening, let it complete greedily, and report the word it plays back.

What happened: coverage

The shape is stark. The first level learns a stable, decodable code for 303 of 364 eligible words, 83 percent: the model recognizes most words by their last character. The middle level learns one. The top learns eighteen. The altitude that should hold the most abstract version of a word holds almost nothing.

The collapse diagnostic says the same thing in another currency. The deep code does not produce hundreds of distinct codes; it produces a handful. With more data the count of distinct codes at the top level falls, from 57 to 37 to 8, before recovering only to 22. A level that collapses to a handful of codes cannot decode many words, so "few distinct codes" reads directly as "few concepts learned." And the broad classes, the dozen categories the words group into, are barely there at the top: one of twelve is decodable at the top level. The model learns words at the bottom and does not lift them into stable higher concepts.

What happened: comprehension against production

The trained reader, asked to recognize and then to speak:

The reader recognizes 296 words and produces 64, a gap of 232, a production-to-recognition ratio of 0.216. It reproduces about a fifth of what it recognizes. Free generation is worse than the count suggests: greedy sampling collapses into a loop and produces no valid words at all, the classic character-model cycle, and the best temperature yields 64 real words against roughly fourteen thousand distinct non-words. Word validity is about one percent at every temperature. The reader recognizes a vocabulary and cannot say most of it.

The playback readout shows why. Prompt the reader with a word's opening and ask it to complete: of the 296 words it recognizes, it plays back 2 to the exact spelling, about one percent. The rest come back as generic frequent syllable stubs. Prompt "pr" and it says "proa"; prompt "ga" and it says "gaie"; prompt "dr" and it says "drea"; prompt "sh" and it says "shie". The reader learned the statistics of what most often follows an opening, not the specific spelling of each word. Recognizing a word (a stable distinct code at its last character) is far easier than producing it (regenerating the exact sequence of characters).

The lesson

The model is a strong recognizer and a weak producer, and the two probes agree on it. Coverage: it learns a decodable code for 303 of 364 words at the first level, almost none higher, and the deep code collapses to a handful of distinct codes. Playback: it recognizes 296 words and reproduces 64, a fifth of what it recognizes, and the codes it learned play back as generic syllable stubs, 2 of 296 to the exact spelling. This is a wall, read two ways. Recognition is cheap; production is dear.

The playback probe is now a standing instrument: point it at the model and read out, in plain text, what concepts it has learned. The first reading is honest and modest. The locked reader is good at the thing our rules are good at, recognizing and priming, and weak at the thing they are not yet built for, generating an exact sequence. That fits the architecture: the top-down wire in this stack carries credit and priming, not generation. It is a recognizer with no clean generative inverse.

The honest caveats

• This is a wall, not a win. Both probes report a limitation. The point is to make the limitation legible and measurable, not to dress it up. The model reads far better than it speaks.
• A controlled vocabulary and a trained reader, two scales. Coverage runs on a controlled corpus with a ground-truth key; playback runs on a reader trained on real text. The numbers sit beside each other but come from two setups. At a smaller scale the gap is even wider (219 recognized against 6 produced).
• Production is read off the comprehension stack. The reader has no dedicated generator. Prompted recall reconstructs each concept by completing its opening, which is why the readout is opening-to-most-common-continuation statistics, not stored spellings.

The frontier

The gap names the work. Recognition is in hand; production is the open frontier, and the program already has its first positive on the internal half of it, the birdsong loop that teaches a producer to say more of what it means with no listener. This round measures the starting line that loop has to move: 64 of 500, a fifth of what is recognized, syllable stubs where words should be. The next question is whether the generation work closes the gap the coverage and playback probes just drew so clearly.

Lineage

Grew from internal self-feedback teaches production, the first positive on the generation frontier, whose comprehension-and-production framing this measures directly; and from from mimicry to production, which drew the line between reading a token for fidelity and producing it for effect.

Thread: generation, and the comprehension-production gap. The mechanism is Minsky's K-lines (reactivate a learned configuration and read it back); the probe is a controlled-vocabulary coverage decoder and a playback dump, two reads of one wall.