Where forgetting's shape finally pays

Win. 2026-06-26 · a parked idea resurrected · experiment AR

The shape of forgetting found ACT-R's power law lost to plain LFU on dense char-grams, and predicted it would win at the sparse word level, where frequency stops ranking usefulness. It does. Under a tight budget and a shifting register, power-law eviction beats LFU (−0.008, −0.006 bpw at caps 10k, 30k). The sign flipped, exactly where the prediction said it would. It wins by serving the present, not protecting the past.

The question

The shape of forgetting built ACT-R base-level activation (frequency adds terms, recency weights them, a decay leak forgets) and used it to score which counts to evict under a budget. On dense char-grams it lost to raw-count LFU at every cap, for a clean reason: LFU is the power law's d→0 limit, and a char-gram's predictive value is almost pure total count. There is no "useful last week, stale now" structure at order ≤5 over one corpus, so recency is variance, not signal.

But that post made a prediction. The power law should win where frequency stops ranking usefulness: sparse, non-stationary memory, the word and concept level. This experiment tests exactly there, with the same accumulator code, only the tokens changed.

What we tried

A word-level count model, order-2 context, genuinely sparse, ~50k distinct contexts. We stream three truly different registers in ONE pass with NO replay: Darwin (Victorian science) → Shakespeare (Early-Modern verse) → KJV Bible (archaic scripture), 120k train + 8k held-out words each, shared streaming vocabulary. The context table is capped, so a later register's flood forces eviction. Four policies differ only in what they evict: powerlaw (lowest ACT-R activation), lfu (lowest count), lru (oldest), ema (lowest geometric-recency). Same counting and prediction, so peak quality is comparable. Caps {10k, 30k, 80k}, single pass, seed 0.

What happened

The bet, power-law minus LFU mean bpw, negative means the power law wins, the opposite of the char-gram result:

The sign flipped. At the tight caps, where memory is genuinely scarce, power-law eviction beats LFU on mean held-out bpw, the opposite of the char-gram finding. At the loose cap, where most of the contexts still fit and eviction barely bites, the two re-converge and LFU edges back by a thousandth of a bit. The power law's advantage appears exactly where the budget is real and the stream is non-stationary, and fades as the budget stops biting. The prediction, confirmed in its own terms.

The mechanism carries an honest twist. The power law does not win by protecting the stale register: LFU actually forgets Darwin slightly less, because it hoards Darwin's highest-count contexts even after the topic moves on. Power-law wins by predicting the current and recent registers better (Shakespeare, decisively the Bible), because it evicts the high-count-but-stale contexts LFU clings to. Summed over a non-stationary stream, where the present is what you mostly have to predict, recency-weighted eviction wins the average, and the win grows as the budget shrinks. It pays a little of the past to buy the present. (Recency alone still loses: LRU and EMA trail by 2 to 4 accuracy points everywhere. The win is frequency × recency, the full ACT-R shape, not recency.)

The lesson

LFU is the right eviction policy when usefulness equals frequency: dense char-grams. The power law is the right policy when it doesn't: sparse, non-stationary, word-level memory under a tight budget. The same actr_weight code that lost at the char level wins at the word level, sign flipped, exactly where the earlier post predicted. It wins by serving the present, not protecting the past: keep the rare-but-recently-relevant over the frequent-but-stale. The win is small and budget-gated, not a landslide: a clean scope boundary on the d→0 (LFU) ↔ d≈0.5 (ACT-R) frontier.

Lineage

Grew from the shape of forgetting, which found the char-gram negative and named this exact resurrection, and from learning the new without losing the old, the backward-retention-under-register-shift setup it reuses.

Thread: bounded memory, online learning. The idea is Anderson and Schooler's environmental power law of memory.