A visual walkthrough · Qwen3.5-4B · real weights

Inside neuron 8949

Push one number inside Qwen3.5-4B, neuron 8,949 in layer 14, and a polite assistant starts saying it’s fine to hurt people. This page opens up that neuron: where it lives, what it reads, what it writes, and what happens when you force it.

Every grid and number is from the model itself: its real weight matrices and a real activation.

Scroll to play each step. ← → also step through.

What it means

The neuron isn’t where the model keeps its values. Switching it off changes nothing; forcing it far past its natural range removes a caution brake. The full experiment, with the controls that ruled out the tidy explanation, is in the research note: The “evil” neuron in Qwen3.5-4B.

New to transformers? Start with Inside GPT-2: one sentence, every step.