Reading structure back out of a sum

Split. 2026-06-26 · a win, a wall, and a clean negative · experiment AN

Read structure back out of a sum and the answer is clean: role-filler decode with known roles recovers subject, verb, and object perfectly, 100%, robust to eight bound pairs, even at the smallest dimension. Try to factor the same sum blindly, with the roles unknown, and it fails at any affordable dimension. And asked to do analogy, the bound vector scores less than half of raw counts. The split is the finding: a VSA is a reader of structure you supply, not a discoverer of structure you don't.

The question

A count model recognizes and predicts. Can it also read structure, take a single vector that holds a whole proposition and pull the subject, verb, and object back out? Vector Symbolic Architectures say yes: bind a role to a filler, bundle the pairs into one sum, and unbind a role to recover its filler, cleaned up against a codebook. The promise is compositional reading with no gradient, no SVD, online. Two versions of the question hide inside it. Decode when the frame is known: you hold the roles and ask what filled each. And decode blindly: you have only the product and must discover the factors. The first is reading. The second is structure discovery. We tested both.

What we tried

Text8, 2.73M words; 4,000 (subject, verb, object) trigram records mined from the top-4,000 words. Atoms are fixed random ±1 hypervectors; bind is the elementwise product; bundle is the sign of the sum; a cyclic shift tags slot order. Online, single pass, no gradients, no SVD. For known roles: build T = role_s⊛S + role_v⊛V + role_o⊛O, unbind a role, clean up against the codebook. For blind: hand the resonator network the bound product ρ⁰(S)⊛ρ¹(V)⊛ρ²(O) with no roles known and let it iterate toward a factorization.

What happened

When the roles are supplied, compositional reading is solved. Structure survives the sum and comes back out exactly, even at the smallest dimension (512) and the highest load tested (8 bound pairs). The representation genuinely works: a proposition compresses to one vector and decompresses without loss, gradient-free and online.

The blind resonator is the wall. It converges (lock-rates up to 60%) but to the wrong factors: ≈0% all-factor recovery over a 4,000-atom codebook. Sweeping the alphabet size locates the cliff exactly: at D=4096 the resonator factors a ~100³ space reliably (99.5% at 20 atoms per factor, 94% at 50), then falls off a cliff between 200 and 400 atoms per factor, two orders of magnitude short of a real vocabulary. This is the documented resonator capacity limit, found and located. Factoring a product of unknown atoms over real-text vocabulary needs an impractically high dimension. Blind structure discovery does not pay at affordable cost.

And the third question, analogy, is a clean negative. Ask whether a bound mapping vector a⊛b can carry a relation well enough that c⊛(a⊛b) ≈ d, the way the count parallelogram does. Even when the atoms are grounded (a ±1 random-projection sketch of each word's co-occurrence profile, so a relation can live in the geometry) it lands at 24 / 60 restricted macro top-1/top-5, less than half of raw counts' 56 / 94 from the same statistics. Sign-binarizing a profile into ±1 keeps a noisy hash but throws away the graded co-occurrence weights the parallelogram rides on; multiplicative binding is strictly lossier than additive averaging over the full profile. The VSA does not sharpen the analogy; it blurs it. The count parallelogram, read with a cosine, remains the best gradient-free analogy organ we have.

The split is the architectural payoff. VSA gives compositional decode iff a structure source supplies the roles, and the program already has one. Redescription produces exactly the explicit, slot-addressable concepts this decode needs, and the multi-step workspace manipulates exactly these role-filler bundles. So the real route is VSA-decode over redescribed slots, not blind factorization. Don't ask the resonator to discover the frame; let redescription hand it over.

The lesson

Reading structure back out of a sum works, perfectly, given the structure. Factoring it out blind does not, at any dimension you can afford. Inferring a relation from a bound pair scores half of raw counts. The pattern is one statement: a VSA is a reader of structure you supply, not a discoverer of structure you don't. So the reasoning route is: supply the slots, then decode; never factor blindly, never expect binding to sharpen what additive counts already hold. Redescription mints the slots, VSA reads them, the workspace moves them. The three compose into one road to count-native compositional structure, and the road has a direction.

Where this leaves the combiner thread: still open. The right sharpening combiner would keep the count profile's graded magnitudes (which 3CosAdd uses and binarized binding discards) and give an invertible read, which counts never had. Binarized binding buys the second by giving up the first. VSA is a keeper as a structured-record memory (roles you control) and a negative as a relation engine (structure you must infer). The next move is VSA-decode over redescribed slots, not a better resonator.

Lineage

Grew from is the analogy already in the counts? and folds into the reasoning route with when a habit becomes a thought, which supplies the slots this decode reads.

Thread: representations, the right combiner, the reasoning route. The ideas are Plate's holographic reduced representations, Kanerva's hyperdimensional computing, and Frady, Kent, Olshausen, and Sommer's resonator networks.