Do concepts emerge as boundaries?
2026-06-26 · investigation closed at 10 variants · experiments CN to CX
The purest form of the standing question: do concepts emerge as the higher tiers of surprise boundaries, words, then phrases, then clauses, then topics, all carved by the same signal one level up? Ten experiments say surprise carves words cleanly and nothing else. Meaning above the word needs a different signal, and the three cheap fixes (compose, scale, share a representation) all fail.
The question
The program's first finding was that surprise marks a boundary: prediction error alone recovers word boundaries from raw characters. The most beautiful version of the whole theory would be that the same trick repeats all the way up. Run the surprise signal over characters and it finds words. Run it over the words it found and it should find phrases. Over phrases, clauses; over clauses, topics. One mechanism, stacked, and the concept hierarchy falls out for free.
This was also the first real test of the node architecture, the synchronous-recurrent shell where every level is the same Column wired bigger, all reading the previous tick's frozen outputs and writing the next. If concepts are the higher tiers of boundaries, this is where it should show, because the shell lets us stack the identical organ at four altitudes with no engine changes. We picked the most Thousand-Brains-pure form of the question (same algorithm, every level) and ran it autonomously across ten grounded variants.
What we tried
Four rounds. The first stacked branching-entropy boundaries (the exact word-level signal) at every altitude (CN). When that carved only words, we stopped asking for the same signal and asked each altitude for its own model-state cue: Bayesian belief-shift for phrases (CP), lexical cohesion for topics on real article boundaries (CQ), event-model predict-and-switch (CO), construction-completion for the stubborn clause tier (CU), a per-altitude cue ladder (CR). Then we tried to make the wins compose into one stack (CS, CW), to make them widen with scale (CV), and to push the clause crack off the floor with a dual-route, sentence-rate-tuned completion signal (CX). Every organ ran unmodified on the runtime. Scored as boundary F1 against held-out structure at a fair tolerance, with a strict 1.5× gate over the random and fixed-stride baselines.
What happened
The altitude map came out clean, and it is mostly a wall.
| altitude | signal | verdict |
|---|---|---|
| word | branching entropy | SOLID. F1 0.83 to 0.98, strict ±1, every corpus and organ |
| phrase | Bayesian belief-shift | WEAK. Beats baselines only at a fair ±2 window, ~1.1 to 1.2×, never the gate |
| clause / sentence | construction-completion | VERY WEAK, fragile. The first cross-corpus crack, but corpus- and tolerance-fragile, segments don't recur |
| topic | lexical cohesion | ~2× at fair tolerance on real <page> boundaries (degrades at large scale) |
Surprise carves words and nothing above them. The higher altitudes yield only weak cracks, and only from signals that are not surprise: a shift in a maintained belief for phrases, lexical cohesion for topics, construction-completion for clauses. None of the above-word cracks clears the strict gate, and none localizes better than about ±2, a precision wall in its own right.
Then the three meta-negatives, each ruling out a cheap fix:
- The wins don't compose (CS, CW). Put word-entropy, phrase-belief-shift, and topic-cohesion in one stack and the phrase tier collapses, even though the same organ scored 0.133 standalone. The controlled diagnostic: the phrase win was a property of the word stream feeding it. Feed it the better word organ and phrase starves. Word and topic compose only because both read the raw word stream independently. The hierarchy is not modular: each tier's cue depends on the representation below it. And a shared, organ-independent lexical representation per level, the obvious fix, does not rescue it: same win/loss at every tier (CW). Non-composition is not a representation artifact.
- Scale isn't the unlock (CV). Neither crack widens. Belief-shift at the phrase level holds ~1.12 to 1.18× across 200k → 3.2M tokens, never reaching the gate; topic-cohesion degrades toward parity with more data. The cracks are weak, not data-starved.
- A shared representation isn't the unlock (CW). The CS fix, ruled out directly above.
The lesson
Concepts emerge as boundaries cleanly only at the lexical tier. Above the word, prediction-error, cohesion, and completion give weak, ~±2-precise, non-composing cracks. Surprise carves words; meaning above the word needs a genuinely different signal (real syntax, a learned grammar), not better wiring, more data, or a shared representation. The three cheap fixes are all spent.
This is a sharp, honest extension of the coherence frontier. The dream of one signal stacking all the way up is dead, and it died the same way three times: compose, scale, share. None of them is the missing ingredient. Whatever carries crisp clauses and events is a different kind of thing.
The quiet structural win underneath: all ten organs wired and stacked on the node runtime with zero engine changes. The synchronous-recurrent shell carried four altitudes of distinct signals without a single modification: the Thousand-Brains same-algorithm shape transfers even though the same-signal ideal does not. The architecture is validated end-to-end, which is what let the next rounds drop counting entirely and still run.
Lineage
Grew from finding where one word ends, the surprise-carves-a-boundary result this stacked, and finding phrases, the first attempt to climb it one level up.
Thread: global coherence, and the surprise through-line. The keeper is a hardened negative (surprise is a lexical signal, not a universal one) and a validated runtime to test the next idea on. The ideas are branching entropy, Kumar and Zacks belief-shift, Hearst's TextTiling cohesion, and usage-based construction grammar.