What changed: the budget bites, binding works, reasoning gets a road

Theory update · 2026-06-26 · the round in one place

Eight experiments, one arc. The bounded-memory rule stops being a prediction and becomes a measured fact. Long-distance binding, the one thing offset-attention couldn't do, is in hand. Compositional reasoning gets a concrete road. And the coherence frontier hardens into a sharper, more useful "no." Here is everything that changed, and where it sends the next swing.

What this round was

A batch of experiments run as one arc, each taking a named result from cognitive science and asking whether it survives translation into counts: online, single-pass, no gradient. Two confirmed standing predictions. One added a primitive the model lacked. Three landed honest negatives that sharpen the theory rather than dent it. Read together they move three things forward at once (the budget, binding, and reasoning) and they harden the one frontier that still won't yield. This post is the synthesis; each claim links its experiment.

(a) CONFIRMED: the bounded-memory rule bites

The rule said the mechanisms that vanish under unbounded scale should return once memory is capped. They do, and now it is measured with a curve. Consolidation flips from a flat −0.006 bpc at unbounded scale to +0.144 → +0.307 bpc as the cap tightens: a monotone function of how hard the budget binds, vanishing exactly when it stops binding. Lossless generalization (consolidation: distill is exact, promote is a fallback) wins broadly; lossy generalization (a coarse topic prior, a mass-spreading category head) wins only as tight as the budget forces; the topic prior stays a clean neutral. So generalization is not optional decoration; it is how a bounded model approximates the unbounded one it cannot afford to be. What survives a budget.

(b) NEW: long-distance binding is in hand

Offset-attention reached attention out of counts but kept two limits: a fixed window and an informativeness that decays with distance. Content-addressable cue retrieval lifts both. It binds a verb to its correct-number subject across an embedded clause: 99.96% where offset-attention's fixed position key tops out at 65% and scores a flat zero past its one modal offset. It reaches by a feature bundle weighted by fan, not by a fixed position. And because activation divides by the fan, the same mechanism reproduces the human agreement-attraction illusion: 0.00% → 2.11% error, appearing only with a recent opposite-number distractor, exactly the one condition a person makes the error. A new primitive: long-distance binding by content cue plus fan. Reaching back by the right cue.

(c) NEW: compositional reasoning has a road

Reasoning was a promise. This round paves it. VSA decode reads structure back out of a sum perfectly (100% role-filler recovery, robust to eight bound pairs, even at the smallest dimension) but only when the structure is supplied; the blind resonator, asked to factor the same sum with unknown roles, fails at any affordable dimension. So don't factor blindly. Redescription supplies the slots, VSA decode reads them, and the System-2 workspace manipulates them: it reaches a two-hop target (acc 1.00) where System 1 and one-step deferral both score 0.00, trapped on the intermediate. Reachability is real. The route is redescription → VSA-decode → workspace-selection. The honest caveat: on a single deterministic chain the workspace's focus-and-inhibition machinery only ties a blind "apply-twice," because that machinery is built to select among competing chains and this probe has none. The named next probe is competing-chains selection. Reading structure back out of a sum · Giving the workspace something to do.

(d) HARDENED: coherence will not yield to stacked state

A persistent situation model (Chambers-Jurafsky event chains plus Zwaan who/where/topic slots) does not predict over long spans. Its +0.55 bpw "everywhere" win was pure smoothing repair: a static frozen unigram beats the live situation by −0.07 bpw on the 99% non-backoff slice, and the situation helps only the same 0.9% backoff slice. That is now three mechanisms (a recency histogram, a Bayesian-surprise slot, a typed schema) all landing on the same 1% island. The frontier hardens into a sharper "no": whatever crosses it must change the 99%, not rescue the 1%. And the methodological lesson is written down: measure a top-down prior against a static prior, never against no prior, or you mistake repair for comprehension. Keeping track of what's going on.

The refinements

Three more results refine the rules without shifting the spine. Power-law (ACT-R) eviction wins at the word level under non-stationarity and a tight budget (−0.008, −0.006 bpw): the sign flipped from the char-gram result exactly as predicted, by serving the present rather than protecting the past. Where forgetting's shape finally pays. Environment-as-memory is a cost arbitrage, not a free lunch: a bounded-internal + external store loses to one bigger internal table at equal budget (evidence fragmentation), and wins only when the page is cheaper than the skull (−0.23 bpc). Writing it down. And phrase pooling works but its addressing blurs the tail: permutation-bound FlyHash addresses pool similar phrases (beats a floored literal on 67% of unseen-form probes) and keep order (×2.29 under scramble), but crosstalk loses the aggregate tail, so it belongs as a backoff layer under the literal table. Phrases that rhyme share their counts.

The synthesis

The theory now reads with three things settled that were open at the start of the round. The budget bites: generalization is the bounded model's only way to stay good after discarding, and the gain is now a measured curve in the cap. Binding works: a content cue reaches across a clause where a position key cannot, and it earns the human interference signature for free. Reasoning has a road: supply the structure (redescription), read it (VSA decode), move it (the workspace), and don't factor blindly; the route is named and its next stress-test, competing-chains selection, is named with it. And the coherence frontier is sharper: it will not be crossed by stacking persistent state, because three mechanisms now prove that stacking only rescues the 1% local prediction already owns.

The frontier

Two probes are named precisely. The multi-step workspace has shown reachability but not selection: the next swing is several competing candidate chains, where suppressing the loud wrong answer should finally make the focus-and-inhibition machinery earn its keep. And global coherence stands where it stood, but with the bar raised: any mechanism that claims it must change the 99% of words local counts already own, not flatter the 1% backoff slice. The reasoning route is supply structure, don't factor blindly; the coherence route is change the 99%, don't rescue the 1%.

The eight posts

• Reaching back by the right cue: long-distance binding by content cue and fan.
• What survives a budget: the bounded-memory rule, confirmed with a curve.
• Reading structure back out of a sum: VSA decode works given structure, not blindly.
• Giving the workspace something to do: the workspace reaches multi-hop; selection untested.
• Keeping track of what's going on: coherence won't yield to stacked state.
• Where forgetting's shape finally pays: power-law eviction wins at the word level.
• Writing it down: externalizing is a cost arbitrage.
• Phrases that rhyme share their counts: phrase pooling works; addressing blurs the tail.