How we work

We publish the failures

A research blog that only shows wins is a sales brochure. This one publishes every experiment, including the ones that lost. The raytracing idea that lost to a bigram has a post. The topic cache that helped a little and then hurt coherence has a post. The boundary signal that scored below random has a post.

The losses carry as much information as the wins. A clean negative result closes a door, names why it closed, and points at the door worth trying next. We write each one with the same care: the question, the attempt, the number, the lesson.

Four standing rules

Underneath the posts are four rules we hold ourselves to. They are not arbitrary. Each is a fact about how a mind works, and together they describe the only kind of intelligence we are trying to build: one that learns as it lives, on a finite budget, the way a person does.

One: everything learns online

There is one hard rule across every experiment, and it is about how learning happens, not what the update is made of. Each piece of data, at the moment it arrives, changes the system: locally, in a single pass, the way a person learns from a thing as they live it. No gradient descent. No backprop. No batch pass that goes back over the data to fit. No k-means, no SVD.

This is a commitment to the regime, not to one mechanism. Counting is the tool we have leaned on most (a model sees a piece of text once, updates its counts, and predicts by looking them up) and an early result was that a plain counter beat two gradient-trained nets. But the rule was never "use counting." Any update that learns on arrival without a backward pass is in bounds: counts, leaky accumulators, online clustering, associative plasticity, prototype formation, binding. The test is simple: does the datum change the system as it is received, with no pass back over the data? And it does not forbid sleep: an offline cycle that replays and reorganizes what was already learned is allowed, because it works on the model's own traces, not by training over the corpus. This is the property a transformer cannot match: it learns from every sentence as it arrives, it never retrains, and a sparse update for new material barely touches what it already knew, so it does not forget.

It also keeps the whole mind inspectable. The counts, the discovered words, the concepts, all of it lives in a Prism document you can open and read. That is the bet underneath the blog: continual, compositional, inspectable learning is the thing a frozen function cannot give you, and it may be the truer road.

Two: we give fragile ideas room to grow

The search space here is enormous. A new idea almost always arrives weak. "No better than a bigram" is the normal first result of a real idea, not a verdict on it. So we do not kill an idea at that gate. We are tender with one in development.

We borrow the stance from Jony Ive: an idea, by definition, is unresolved. If it were finished, it would not be an idea. So we judge it by trajectory, not by level. We set a budget of ten to twenty real variations before any decision to stop, because in a high-dimensional space the wins compound late.

And before we shelve anything, we check the other dials. An idea ties on top-one accuracy but cuts perplexity threefold. An idea loses on clean text but wins the moment the input is noisy. An idea loses overall but wins decisively on the slice where the local context has run out. Each of those would have been thrown away if we had only read the headline number. So before shelving an idea we measure calibration, robustness, generalization, rare-context behavior, and transfer. A fragile idea usually wins first on a dimension you were not headlining.

A shelved idea goes to a graveyard, not a trash can. We record why it stopped and the step it died at, so it can come back when a complementary piece arrives.

Three: the model runs on a memory budget

A mind in the real world has finite memory. It cannot keep every count forever. So neither can this model. We impose an explicit memory budget on every experiment: no unlimited tables, no keeping every context it ever saw.

This rule is what makes the project honest. With infinite memory, more data subsumes any mechanism that merely re-predicts what raw counts already hold, and the clever pieces look redundant. A budget reverses that. When you cannot keep every count, you must throw most of them away, and the only way to stay good after throwing them away is to have generalized first. So the mechanisms that looked redundant at unbounded scale are exactly the ones a bounded model needs.

There are three ways it copes, and we build and measure all three. It generalizes: it compresses many instances into reusable concepts and constructions, so it can drop the instances without losing the prediction. It sleeps: an offline consolidation pass prunes lossy counts, distills the specific into the generic, and merges duplicates to stay under the cap. And it uses the environment as memory: it writes durable, rare knowledge to an external store it can re-read, the way we write things down, so the internal store stays small and fast while the world holds the long tail.

Bounded memory is what forces generalization. Generalization is the point. The constraint and the goal are the same thing.

Four: human cognition is the guide

Human intelligence, flaws and all, is the only thing that achieves general intelligence under exactly these constraints: online, single-pass, finite memory. So we treat it as both the existence proof and the design oracle. When we are unsure how a piece should work, the right question is not "what is optimal?" but "how does a person do this?" The answer is usually buildable on counts, because the brain is not running gradient descent either.

We follow it including its quirks, because under real constraints those quirks are adaptive priors, not defects. Availability is recency-weighted counting. Representativeness is prototype matching. Anchoring is sticky context. Good-enough is stopping at the first confident answer. Forgetting is the memory budget itself. A bias that hurts on a contrived puzzle is often the right bet on the real distribution, so we keep it and check before we "fix" it.

This rule also names the gap we have not closed. Everything built so far is System 1: fast, parallel, associative, counts voting, instantly, to recognize a word or feel a topic shift. What we have not built is System 2: slow, serial, working-memory-bound, deliberate, the system that follows a chain of steps, holds a few items and manipulates them, and checks its own work. A count-native System 2, a small capacity-limited workspace that reasons over System-1 outputs under metacognitive control, is now a primary direction.

The post template

Every post follows the same shape, so each one reads in a couple of minutes:

• The question, and why it mattered. What we wanted to know, in plain words.
• What we tried. The setup, without the plumbing.
• What happened. The one number, or the small table, that carries the result.
• The lesson. The honest takeaway, the negatives included, and what it changes about where we go next.