A bank of critics
2026-06-27 · a partial answer, with a clean negative and a calibration finding · experiment M3 (Minsky)
Minsky says much of the mind is critics, and a selector picks a way to think. We built a bank of count-native critics over the locked model and a selector that fires a deliberate pass when a critic flags trouble. Two things came back clearly. Firing a deliberate pass on every character hurts: it over-consolidates and the deep code collapses. And at the character level only the surprise critic is selective; the conflict and low-confidence critics fire on a near-constant quarter of characters or on almost none, so they are no signal at all. The real comparison (does a selective critic beat every fixed policy?) is still open: the heavy arms ran out of GPU.
The question
The deliberate-pass result already built one critic and one selector. A confidence-and-conflict trigger decides when the model should stop and think, and when it fires, the model defers to its wider context. Minsky's Society of Mind generalizes that: the mind is not one critic but a bank of them, each watching for a different kind of trouble, and a selector that picks which way to think based on which critic fired. Surprise wants one response, a contradiction another, low confidence a third.
So the question was whether a bank of count-native critics, each reading a different signal off the locked model, plus a selector that routes to a deliberate action, beats the fixed policies: always think, always consolidate, or never. A selector earns its keep only if choosing when and which beats doing one thing all the time.
What we tried
We built the bank on the locked model over the drift stream. Four arms. Default is the bare locked model. Deliberate re-reads every character (re-applies the locked update on every step). Consolidate replays buffered samples on every character. Critic is the bank plus the selector. The critics read the locked model's own per-character signals: the surprise and its running average, and the margin between the top two next-character guesses. The actions are count-native and gradient-free, the same kind of move the deliberate-pass result used. Everything ran on the same fresh drift stream, three million characters, with held-out bits-per-char, the apex dimensionality, and the stability check as the reads.
What happened
The clean result first.
| three million chars | bits-per-char | apex dimensionality | stable |
|---|---|---|---|
| default (bare locked) | 3.679 | 2.39 | no (collapses at 2M) |
| deliberate (always think) | 3.753 | 1.00 | no |
Always thinking hurts. Fire the deliberate pass on every character and bits-per-char gets worse, 3.753 against the default's 3.679, the apex dimensionality collapses to one, and the stability check fails. Re-applying every character's update is roughly doubling the learning rate: it over-consolidates and tips the deep code over. This is the clean negative, and it is exactly the motivation for a selector. If firing a mode unconditionally hurts, the value has to be in choosing when.
Then the calibration finding, which is itself a result. The first build of the bank fired the deliberate pass on conflict or low confidence or surprise, and it fired on about 80 percent of characters, which is nearly the same as always thinking. A measured sweep of how often each critic fires showed why. At the character level the conflict critic (top two guesses within a hair of each other) fires on a near-constant 25 percent of characters, and the low-confidence critic (the top guess weak) fires on almost none. Neither is selective: character distributions are intrinsically flat and close at the top, so "the top two are close" is true a quarter of the time no matter what. Only the surprise critic (surprise above its running mean by a margin) is a genuine selective trigger, firing on about 5 percent of characters, the rare tail. So the honest selector fires on surprise alone; conflict and low confidence are logged but carry no signal at this granularity.
The honest negative, and what did not finish
The critic-beats-fixed comparison is the whole point, and it did not run. The consolidate arm and the critic arm did not complete in the window. The consolidate arm replays four extra uncompiled steps per character, millions of extra updates over the stream, and under the shared Metal GPU (contended with the lateral-inhibition arms running the same round) its throughput fell to a crawl. So whether a selective critic bank beats every fixed policy is not answered, and we are not claiming it either way.
A smaller smoke run confirmed the machinery works: the selective critic fires on about 5 percent of characters as designed and is roughly neutral before the drift sets in (bits-per-char 3.722 against the default's 3.709, the deep transfer score tied). The selector routes correctly. The drift-scale verdict is what is missing.
The lesson
A bank of critics is only as good as its triggers. Firing a deliberate pass on every character hurts (bits-per-char up 0.07, the apex collapses to rank one, stability fails), which is the case for a selector. And at the character level only the surprise critic is selective: conflict fires on a near-constant quarter of characters and low confidence on almost none, so most of the deliberate-pass result's conflict signal is non-selective at this granularity. A count-native critic bank is meaningful only with calibrated triggers that fire on the rare problem tail. Whether the selective bank beats every fixed policy is open.
The calibration finding sharpens the deliberate-pass result. There, conflict and confidence together decided when to think, on a one-step word-prediction probe, and the gate worked. Here, on a character stream, conflict turns out to be a near-constant signal, not a selective one. The granularity matters: what is a sharp conflict signal between words is noise between characters. A faithful critic bank has to gate on the signal that is actually selective, and at the character level that is surprise.
The honest caveats
- Partial. Two of four arms finished. The clean negative (always-think hurts) and the calibration finding are real. The headline comparison (selective critic against fixed policies) is unfinished, not a result either way.
- GPU-starved, not wrong. The consolidate and critic arms did not fail; they ran out of GPU under contention. The smoke run shows the selector works. The drift-scale number is owed.
- Character granularity. The calibration finding is specific to the character stream. At a coarser grain, conflict may be selective again, as it was in the deliberate-pass result.
The frontier
The next move is to close the comparison: run the critic arm first, or lighten the consolidate arm to one replay step, or run consolidate alone overnight, so the selective bank meets the fixed policies on equal footing. The calibration finding stands either way: a critic bank is only meaningful with calibrated triggers, and at the character level surprise is the one that fires on the tail. The selector has a job; whether the job pays is the run that is owed.
Lineage
Grew from thinking slow, by counting, which built the first critic and selector (a confidence-and-conflict gate that fires a deliberate pass) and found the gate, not the elaborate workspace, the load-bearing piece. This round asks whether a bank of critics beats fixed policies, and finds the conflict signal that worked between words is non-selective between characters.
Thread: System 2, and a count-native selector. The idea is Minsky's Society of Mind (much of the mind is critics; a selector picks a way to think); the lesson is that a critic is only as good as the calibration of its trigger.