By Luxia (@kataletheia), with Claude (Anthropic) · compute and hosting by Anima Labs
Each point is one FFN neuron at one depth: 294,912 in all. They are placed by what they respond to in 24M tokens of the model's own training text. The model, its training run and every measurement here come from one project, from random initialisation onward.
Every neuron was read three ways: which passages make it fire hardest, which tokens its output favours, and what the model writes when that neuron is pushed. Neurons whose firing contexts mean similar things sit together. Agents then named every group and sorted them into families.
Brightness is each neuron's average attribution across all eight binding families; ice pushes toward the right answer, ember away from it. The rest of the page has the numbers, and a map to explore.
Switch the layout to see the same neurons placed by meaning, by depth or by their weights; points glide between layouts. Click a point to open its group. Language-model agents wrote the group names and neuron labels from the three readings, and graded each one by how well the readings match. Treat them as drafts.
Removing 702 interferers moves who said what in held-out multi-speaker scenes. The margin for the right speaker rises by +.33 nats (95% interval +.16 to +.50). The 702 are the union of the interferer sets chosen on bare items of ten binding tasks. Random sets of the same size move the margin by under .02. Random sets 8 to 15 times larger, matched to the union's damage at half dose, reach at most +.08 at full dose. At half dose the union's effect is small (+.05), and one of those large random sets matches it.
246 neurons stay in a top-100 set when the items come with four balanced worked examples: 133 interferers and 134 supporters, from 315 of 1,400 places. Of these, 21 support one task and interfere with another. The circuits above come from bare items, with no worked examples. Attribution was run again on the same items in the new format. For each of seven battery tasks, the top 100 supporters and the top 100 interferers were compared between the two formats.
The wiring among them holds at about twice the rate of unrelated maps. Attribution also scores each link between two of these neurons: how much one neuron's activity feeds the other's effect on the answer. The links among the 246 were measured for attribute binding, dialogue and code variables. Between 23% and 27% of them pass the same test in both formats: same sign, and strength above the task's median. Two unrelated maps would give 12.5%. For dialogue and code variables, the links are about as stable across formats as between two halves of one format's items. Attribute binding rewires: its links have a rank correlation of .17 across formats, against .33 to .47 between two halves of one format. Whole maps move more. Under balanced worked examples, the maps for attribute binding, dialogue, adjacency and code mutation change well beyond their own noise. The maps for code variables and answer change about as much as their noise allows.
distinct neurons, from 315 of 1,400 places
against 12.5% for two unrelated maps
Each dot is one of the 246 neurons, placed by depth (left to right over the 72 effective layers) and by the task where it held. Each line is a link that passes in both formats. Hover a neuron for its name; click it to pin its strongest links. All eight hubs of the first map hold, among them 19/3306, the pass-1 neuron whose removal lifts attribute binding by +.57 nats. With balanced worked examples, the word-problem chain that starts at 14/1390 keeps its links, and the net push of the 246 neurons on dialogue goes from −.43 to +.06.
Depth bands: prelude (0–18), the looped core's first pass (19–38), the same weights on the second pass (39–58), coda (59–71). Rows group neurons by the task where they held; "shared" means two or more tasks. Links: measured attribution graph on 40 held-out items per task in both formats.
Neuron 3306 at position 19 is a causal hub against attribute binding on pass 1. Setting it alone to its average raises the attribute margin by +.57 nats with balanced worked examples, and by +.34 on bare items. Ten random single neurons at the same position move it by .013 or less. With balanced examples, removing it slightly lowers the dialogue and code-variable margins (−.014 and −.016). Positions 19–38 and 39–58 run the same weights, so every neuron of the looped core has one copy on each pass. The pass-2 copy at position 39 has no measurable effect: it moves each task's margin by .013 or less. Attribution had ranked both copies among the neurons with the strongest links to later neurons, and the knock-out separates the two.
nats on the attribute margin, with balanced worked examples
nats on any task's margin: no measurable effect
Share of each family's attribution at each of the 72 effective positions. Positions 19–38 and 39–58 are the same weights run twice. The question tokens at the end of each item carry most of the attribution; the names themselves carry little.
| Family | Category | Neurons | Readings agree |
|---|
These neurons were characterised by hand before the map existed (firing screens, steering, forked log-probabilities). The map was built without those results. The loop gives a free test: a neuron's two passes share weights, so if the map tracks function, the two copies should land in the same group.