Scalpel: steering an LLM from a single feature
Scalpel demonstrates, with numbers and scientific controls, not screenshots, that a single interpretable feature inside a language model causally controls one specific behaviour. This is mechanistic interpretability research: understanding how a model "reasons", so we can make it transparent and controllable instead of a black box.
Star on GitHub
A research project that makes a language model transparent and controllable: it finds an interpretable "knob" inside the model and proves, with rigorous measurements, that turning it changes the model's behaviour in a predictable way.
The problem
Language models are black boxes. They answer, but nobody knows why they say a given thing, and that is a serious problem when AI enters a real business process: how do you trust a system you can neither inspect nor correct? Mechanistic interpretability is the field that tries to answer this: open up the model, locate the internal "circuits" and understand what they represent.
Scalpel tackles the most concrete question of all: does a single internal feature really, causally, control a specific behaviour? Not "it seems so" from one example, but demonstrated with numbers and controls.
What Scalpel does, in plain terms
- It finds an interpretable knob. Using a Sparse Autoencoder (a technique that decomposes the model's internal state into human-readable features), Scalpel locates the feature that corresponds to a concept, for example "references to dogs". It confirms this with Neuronpedia's public labels.
- It turns it. During generation, it adds the direction of that feature to the model's internal state, with an adjustable intensity (even negative, to suppress the concept).
- It measures the effect instead of narrating it. An intensity sweep, a concept score, and, above all, the controls that make the result credible.
The results
Dose-response curve. As intensity increases, the concept grows in a clean and predictable way, peaking at the optimal dose; push further and the text loses coherence (perplexity rises). There is a measurable balance point.
The control that makes it credible. A random direction of the same intensity produces an effect of 0.000 at every level: pushing the internal state at random never creates the concept. This is what separates a scientific result from a lucky screenshot, the feature's effect is causal and specific, not an artefact.
Specificity. Turning the knob of the target concept moves only that one: three unrelated concepts stay flat at zero while the target rises.
Everything is reproducible with a single command, with fixed seeds, automated tests and CI. And there is a note of intellectual honesty in the README: against a strong baseline (mean-difference) the results are reported as measured, not inflated.
Why it matters (for a company too)
This is not an academic exercise for its own sake. Those who understand how these models really work build different AI:
- Verifiable AI, not "magic". Knowing that a behaviour has a precise internal cause means being able to inspect it, explain it and, if needed, correct it. It is the opposite of a black box.
- Control and compliance. Transparency and controllability are exactly what the AI Act and enterprise clients ask for. Interpretability research is the technical foundation of that promise.
- Real competence. It is the difference between those who use AI and those who understand it. And it shows in the final product.
Learn more
Code, method, results and a reproducible notebook are open source (MIT).
Rayo Consulting research · github.com/dylanpatriarchi/scalpel