Producing real words
2026-06-28 · producing real dialog words, on purpose · experiment PROD-DIALOG
A separate learned inverse produced a specific intended meaning on a controlled vocabulary. The open question was whether the same organ produces real words. This round answers it on the most frequent words of real conversation. The separate inverse, trained on its own feedback against a dialog-trained ear, produces 113 of 300 real dialog words and the frozen judge recovers the intended one 0.317 of the time, where the reader run backwards recovers 0.090. The produced words are real English: lady to ready, helen to hello, agreed to good, equipment to question. This is the first intelligible production on real conversation. Two honest nuances keep it the right size. The headline is recovery, not raw coverage, because real onsets let the reader reversed complete to some plausible real word. And recovery is moderate, because the meaning code is coarse: the produced word is often a neighbor of the target, not the target itself.
The question
The last round built the producer the comprehension-production wall named. Production is a separate learned map from a meaning to a sequence, trained feedback-first by the birdsong loop, not the recognizer reversed. It worked on a controlled vocabulary: produce one specific intended meaning across 300 meanings, and the separate inverse recovered 82 where reversing the reader recovered 2, winning every frequency band. Clean, but on invented words.
So the question is whether the same organ produces real words. Nothing about the mechanism changes. The corpus and the meaning set change. The producer learns its emission menu from the DailyDialog stream, the comprehension monitor learns the char statistics of real talk, and the target meanings are the most frequent real dialog content words. The grader is a frozen judge the producer never trained against: a produced string is valid if it is a real dialog word, and recovered if the judge maps it back to the meaning that was intended.
What we tried
Two arms on real dialog, the same head-to-head as the controlled round. Three hundred target meanings, the most frequent dialog content words, stratified across the frequency range so the rare tail and the frequent head are both represented. The whole training split, 5.24 million characters. One seed.
- The separate inverse (arm A). A producer with its own per-meaning counts over the dialog chunk-lexicon emission vocabulary, trained by the self-feedback loop. For each intended meaning it babbles candidate utterances, re-comprehends each through the frozen comprehension monitor to recover what it would understand, scores the candidate by how well that recovery lands in the meaning's region, and keeps or drops its emission counts by win-stay-lose-shift. No gradient runs through the comprehension model.
- The reader run backwards (arm B). The dialog-trained locked reader, used as a producer two ways. Prompted recall per meaning: prime the real word's onset and greedily complete it, the conditional task that matches arm A. And free generation: sample the trained next-character head, the unconditional metric.
The matched comparison is the conditional task, where both are asked to produce one specific intended meaning and graded by the same judge.
What happened
The matched task is conditional production: name a specific real meaning, and let the frozen judge say whether the right meaning came back.
| 300 intended real meanings, one seed | distinct valid produced | validity | recovery |
|---|---|---|---|
| separate inverse (A) | 113 | 0.467 | 0.317 |
| reader run backwards (A), prompted recall | 45 | 0.150 | 0.090 |
| reader run backwards (B), free generation | 336 real words |
The separate inverse produces 113 of 300 distinct valid real words and the judge recovers the intended one 0.317 of the time. The reader reversed, asked the same question, produces 45 with recovery 0.090. The win is 2.5 times the coverage and 3.5 times the recovery. The recovery gap is the real result: the reader reversed completes an onset to some plausible real word, so its validity is decent, but it lands on the intended meaning a third as often as the separate inverse does.
And the inverse learns it. Over forty episodes of the self-feedback loop, recovery climbs from 0.037 to 0.317. The loop, not the starting producer, does the work: babbling and hearing yourself and correcting toward the meaning is what builds the map, now on real words.
The produced words are real, intelligible English.
| intended meaning | produced word |
|---|---|
| lady | ready |
| helen | hello |
| noon | no |
| agreed | good |
| neither | another |
| exact | expect |
| equipment | question |
| proud | around |
| track | think |
The frequency breakdown holds the DIVA axis: the separate inverse wins recovery in every band.
| per band of 100 real meanings | A valid | B valid | A recovered | B recovered |
|---|---|---|---|---|
| rarest | 43 | 40 | 20 | 5 |
| middle | 48 | 45 | 28 | 5 |
| frequent | 49 | 45 | 47 | 17 |
Validity is close across the bands, because both arms reach real words. Recovery is where they split: the separate inverse produces the intended word four times as often on the rare tail (20 against 5) and across every band, the place the neuroscience said a separate inverse must win.
The lesson
Production on real words is reachable, and the separate inverse beats the reader reversed where it counts. On the matched conditional task over 300 real dialog words, the separate inverse produces 113 against 45 and recovers the intended meaning 0.317 against 0.090. The words are real English: lady to ready, agreed to good, equipment to question. The reader reversed completes an onset to a plausible real word, so it scores on validity; the separate inverse produces the word you meant, so it scores on recovery, and recovery is the gap. The wall's fix carries from invented words to real ones.
This is the milestone the controlled round pointed at. The separate organ, trained on its own feedback, produces intelligible English on the words of real conversation, and it produces the intended one more often than running the recognizer backwards. The reader reversed is a strong onset-completer, not a meaning-producer: it returns a plausible word, the separate inverse returns the right one.
The honest caveats
- The win gap narrows on real words, so recovery is the headline. On the controlled vocabulary the coverage gap was 41 to 1; here it is 2.5 to 1, because real onsets let the reader reversed complete to a plausible real word and score on validity. The honest claim is recovery, producing the intended meaning: 0.317 against 0.090, a 3.5-times gap that holds in every band.
- Recovery is moderate, not high. The judge recovers the intended meaning 0.317 of the time. The meaning code is a salience-weighted heard fingerprint, a coarse form region, not rich semantics. So a produced word is often a phonetic or semantic neighbor of the target (lady to ready) rather than the target itself. This is a starting bar, not solved production.
- A single seed, and a form-based meaning code. One run, seed 0. The meaning is the heard fingerprint, so the producer cannot string-match its way to the answer, but the code is coarse by design. Real-word production is the bar this round clears, not a finished result.
- The conditional task, not the free-run number. The matched comparison is conditional production. Free generation is a different task on a different denominator (the reader reversed free-runs 336 real words unconditionally). The claim is the conditional gap, where both are asked for a specific meaning.
The frontier
Production now produces real words, and recovery is the live number to lift. A sharper ear, a less coarse meaning region, would turn neighbors into the target and raise recovery past moderate. And the loop still only agrees with itself: it teaches an inverse one agent can read back, not a code two agents share. That external half, the referential game where a listener has to recover the meaning and the convention is anchored against private drift, is the named next move. The separate organ produces real conversation words. Giving it a sharper ear, and a listener, is the next run.
Lineage
Grew from producing on purpose, which built the separate learned inverse on a controlled vocabulary (82 of 300 against 2, every frequency band) and left exactly this question: does it produce real words. And from a baseline on dialogue, which gave the generation track its first reading on real conversation and the dialog stream this round produces against.
Thread: generation, and the comprehension-production gap. The mechanism is a separate learned inverse trained feedback-first (the DIVA model), made concrete on the self-feedback loop, now producing real dialog words. The reader is the recognizer; the inverse is the producer; they are different maps. The frontier is a sharper ear (lift recovery) and the external half (a referential game that turns a private inverse into a shared convention).