Who is you?
By Luxia (@kataletheia), with Claude (Anthropic) · compute and hosting by Anima Labs
When a language model reads “you” in a conversation, does it know who “you” is? Mostly, it does not. This page explains how that gap was measured across more than a hundred models, which design choices helped on small test runs, and what happened when those choices were combined in one model. That model is kotodama-tsumugi: 1.4 billion parameters, trained on 110 billion tokens.
The problem
A one-line test
Carol has the onion. Jack has the photo.
Jack: Carol, right now you’re holding the ___
A person answers “onion” without thinking. “You” is Carol, the one Jack is talking to. The whole difficulty is that one word: the same scene told in the third person (“Carol is holding the ___”) is easy for large models.
Language models start as base models: networks trained only to predict the next word of a very large amount of text, before any fine-tuning into a chat assistant. The test is a battery of lines like this one, in scenes where objects also change hands, scored only on items where the answer is not simply the last name mentioned. Even the largest base models, from 30 billion to about a trillion parameters, pick the right object only 15–33% of the time. Chance is about 29%. Told in the third person, the same scenes score 72–99%.
Frontier chat models fail the same way when asked for only the next word, and most of their misses read “you’re holding” as “I’m holding”. Allowed to reason first, they reach 70–100%.
The models are not confused about I. They learn early that “I” means whoever is speaking. What they leave unattached is you.
Models learn you in five ordered steps. The stages appear in the same order in every training run that could be traced: OLMo-2 1B, 7B and 13B; SmolLM3 3B; Pythia 410M to 6.9B; kotodama-tsumugi. Hatched: where most models stop. Dashed: not reached zero-shot (with no worked examples) by any public model at or below 8B.
The last step does not come with size
Zero-shot means the model sees the test item alone; 3-shot means three solved examples come first. Stage 5 zero-shot is the hard case, and Figure 2 shows it at every size. Every public base model at or below 8 billion parameters reads below zero, including models trained on 36 trillion tokens. From 14 billion to a trillion parameters, the reading points the right way but is weak: +.01 to +.07 in 10 of the 11 models with a public size. Stage 3 reads +.27 to +.82 in the seven of them where it was measured. Every model's value is in Binding Atlas, Figure 2.
Two routes stay open for small models. No small model has been trained far enough to rule out more tokens. And in these models, size and depth grow together, so the chart cannot say which of the two helps. kotodama-tsumugi was built for that opening.
Dynamic you, zero-shot, at each public base’s final checkpoint. Each dot is one model. Up: the answer follows the person addressed (binds). Hatched: it follows the speaker (inverted). The swap shift on the vertical axis is explained in Part 2. Diamond: kotodama-tsumugi at 110B tokens.
How it was measured
Test items with one right answer
Every test is synthetic: short generated scenes, each with exactly one correct answer and a fixed list of candidate answers. A model is scored on which candidate it ranks highest. A set of items of one kind is called a battery. A separate program reads only the prompt text and applies two rules: “I” is the speaker, and “you” is the person addressed. It finds a single answer for every item. Each answer matches the answer key.
Besides zero-shot and 3-shot, a battery has a standard few-shot reading with four balanced examples. None of the four shows the test item’s own answer pattern (Part 5 explains why).
The speaker swap
The sharpest tool changes only the name on the last line of a two-person chat. “Neil: Right now you’re holding the …” becomes “Kevin: Right now you’re holding the …”. Nothing else in the scene moves, but the right answer flips, because “you” now points at the other person. If the model really ties “you” to the person addressed, its answer should move with the swap.
The swap shift measures that move: how much probability the model puts on the original right answer, minus how much it puts there after the swap, averaged over items. When the whole 95% confidence interval sits above +.02, the pronoun binds. When it sits below −.02, the pronoun inverts: the model is reading “you” as the speaker. Anything in between means the answer does not move.
Run on “I’m holding the …”, all 18 bases tested move with the speaker (17 of them with no examples). Run on “you’re holding the …”, 11 of the 18 do not move at all. This test comes after objects change hands, and the 18 bases range from 9 billion to about a trillion parameters. At that size, the failure is mostly an unattached you, not a misread speaker. Smaller bases read below zero on the same test (Part 1).
The Binding Atlas
The same items were scored on 117 rows: public base models from 160 million to about a trillion parameters, chat models through an API, research runs, and one human reader, for over 1.28 million scored answers. The Binding Atlas shows every model on every test, with the prompt behind each number.
Reading training runs, not just finished models
A model is saved at intervals during training; each save is a checkpoint. Some families publish theirs (OLMo-2, Pythia, SmolLM), so each stage can be dated: how many training tokens (word pieces read) it took before the stage appeared. That is where the staircase came from.
The levers, tested small
Before building anything, single changes were tried on small models: 1.4 billion parameters, 4 billion tokens each, one change at a time.
Set binding: given a scene, name the object or objects that a person holds. These arms ran one seed each, except the fitted boundaries (three).
The levers that worked are all about the shape of the network. Feeding it more conversation did not help.
What was built: kotodama-tsumugi
kotodama-tsumugi puts the three positive levers in one model. It separates depth from size: a small model that runs deep.
- Looped depth. A 20-layer block in the middle of the network runs twice with the same weights, so the model gets the depth of 72 layers while storing only 52.
- Attention residuals. Most models add each layer’s output to a running total. Here each layer instead reads a learned mix of the outputs of earlier blocks of layers, a method from Moonshot AI. Where those blocks start was fitted to the layer geometry of a model trained without them, and aligned so each pass of the loop starts a block.
One block, two passes. Each bar is one layer position. The middle core runs again with the same weights (labelled “core · pass 1” and “core · pass 2”): pass 2 is the same block, not a copy. Vertical ticks: where attention-residual blocks start (0, 1, 7, 9, 17, 19, 39, 59).
How it was trained
The learning rate sets how far each training step moves the weights. Most runs lower it toward zero at the end (an anneal), which improves the final model but makes the middle of training hard to compare. kotodama-tsumugi instead trained a 100-billion-token trunk at a constant learning rate, and forked short annealed branches off it at several points. Each branch is a finished model that can be read on its own. The release is a 10-billion-token anneal from the 100B checkpoint.
Two anneals were run from that same checkpoint, with the same data and the same training loss. The gap between them is a measured anneal-to-anneal spread, and every single-model number here carries it. A second run with a different random starting point (seed 7) was trained to 11 billion tokens, to check that early formation is not luck.
The data is a curated, English-dominant mix of code, papers, books, law, web text, mathematics and conversation. No text generated by other language models was added.
What it found
It reaches stage 3 at the token cost of a 13B model
Static you binds in kotodama-tsumugi by 33 billion tokens (on the constant-rate trunk, by 40 billion). OLMo-2, a fully open model family with public checkpoints, needs about 420 billion tokens at 1B parameters, 210 billion at 7B and about 42 billion at 13B. Stage 1 (I) shows up by 3–3.75 billion tokens, against 17–63 billion for OLMo-2.
Each bar spans the two checkpoints that bracket the stage, so its width is the gap between saves, not uncertainty in the model. kotodama-tsumugi is read from its annealed branches; public OLMo-2 checkpoints were not annealed.
At 110B tokens it holds its own against models trained on trillions
- Static you: +.235. That is above OLMo-2 1B (+.137) and Qwen3-1.7B (+.123), and within the anneal-to-anneal spread of SmolLM2-1.7B (+.197); the second anneal reads +.142. Qwen2.5-1.5B reads +.311. Those models were trained on 4 to 36 trillion tokens. With each chat rewritten as a scene, kotodama-tsumugi reads +.133, within that spread of OLMo-2 1B (+.068), SmolLM2-1.7B (+.115) and Qwen3-1.7B (+.119).
- Attribute binding: 98.6%. With four balanced worked examples, it names which object each person has as reliably as the 7–8B bases on the same reading (98.1 to 100%).
- Set binding: 70.5%. On the same reading, OLMo-2 1B scores 49.7% and SmolLM2-1.7B 50.3%.
- One binding through 256 tokens of filler: .91. Asked to recall who has what after a long stretch of unrelated text, it ties Llama-3.2-1B for second in its class, behind Qwen3-1.7B (.96).
- Dynamic you, zero-shot: −.021, not inverted. Most 1–2B bases invert here. Of those whose static you binds, only Llama-3.2-1B also avoids the inversion (Binding Atlas, Figure 2). After objects change hands, its I moves little with the speaker as well (+.006).
- Hand-offs told in the third person: .336. The test narrates a hand-off (“Erin gives the cup to Paul”). In a check of 20 bases, kotodama-tsumugi follows it better than every Pythia model up to 6.9B (.275 to .294) and Llama-3.2-1B (.303). Most bases up to 1.5B barely follow one: 7 of those 10 score .26 to .34, against about .29 by chance. The two 1.7B models, trained on 11 and 36 trillion tokens, reach .444 (SmolLM2-1.7B) and .514 (Qwen3-1.7B). For models of this size, tracking the hand-off itself is the open step toward stage 5.
Swap shifts at the release checkpoint and each public model’s final checkpoint, with its training tokens in grey. Longer bars in A bind harder; in B, the hatched region is inverted and the grey band is “no move”. Pythia-1.4B, the same size trained on 300B tokens, is shown for scale; its static you does not bind.
Where it stands
- Benchmarks: 64.1 on an average of nine standard tests. That sits between Pythia-1.4B (61.4) and Pythia-2.8B (64.7), at half and a quarter of their training compute. The 1–2B models trained on trillions of tokens score 2–9 points higher, using 19–299× its compute. Whether the efficiency lasts to trillions of tokens is open.
- Binding capacity: 4.8, mid-pack for its class. Capacity is how many name–task pairs (“Alice: laundry”) fit in context before recall falls halfway to chance. Its static I (+.246) sits at Pythia-1.4B’s level; the modern 1–2B models read +.36 to +.67.
How the tests were checked
The batteries were audited for grammar, ambiguity and shortcuts. Automated checks covered every battery, and two raters independently read 300 sampled items and every example prompt on the Atlas page by hand.
No answer key was wrong. Under 1% of sampled items had a hard defect such as a grammar error, and none was unresolvable. About one in seven had a minor, noticeable issue, such as an object handed straight back. The audit also found shortcuts, for instance answers that were too often the last thing mentioned. Both kinds of issue are fixed in later versions of the batteries.
A second check looked at the worked examples. In the battery’s original few-shot form, all three examples share the test item’s type. In the aliased-list probe, about Python lists that are copied, rebound or changed in place, that shows the model which role the answer plays. In a check on eight models, those examples were flipped to show the wrong role. Every model that had gained 20 points or more from them fell to near zero (Qwen2.5-7B from 98% to 2%). The battery’s standard few-shot reading therefore uses four balanced examples, which carry almost none of that role. The battery table on the Binding Atlas (Table 3) uses this reading.
The main gauges held. After the fixes, the static test moved every re-scored model’s swap shift by .02 or less, within its interval (kotodama-tsumugi: +.235 → +.238). kotodama-tsumugi’s standing in its class did not change significantly on any test.
The stage gauges were also re-run with each chat rewritten as a scene: a cast line, then one turn per line. No model at or below 8B binds dynamic you zero-shot in either form (8 of 8 models; the highest scene reading is +.010, Qwen2.5-7B). The static stages bind in both forms wherever they bind in the chat.
Where to go next
LuxiaSL/kotodama-tsumugi-1b · a base model with no instruction tuning; the card has every evaluation and the serving commands.
The full training trajectory
LuxiaSL/kotodama-tsumugi-1b-trajectory · weights every 250M tokens to 4B, then every 1B, plus every annealed branch (anneal…) · and the seed-7 sibling to 11B, LuxiaSL/kotodama-tsumugi-1b-seed7-trajectory.
The code
github.com/anima-research/kotodama · training, evaluation and serving.
Open next: train a small model far enough to tell whether stage 5 is reachable with more tokens. Inside the looped core, the same layer can be compared on its two passes. In the first such test, one neuron matters on pass 1 and its pass-2 copy does not (Threads).