The problem
When a language model answers, it sounds equally confident whether it knows the fact or is inventing one. The text carries no tell. Every hallucination detector I had seen worked from the outside: ask a second model, sample many times, check a knowledge base. All of them cost extra calls and none of them look at the one place the truth actually lives, the model's own activations.
The opening
In July 2026 Anthropic published Jacobian-lens research: a way to transport a mid-network activation into the output vocabulary and read what that activation is disposed to make the model say. The papers shipped with fitted lenses for open models. I read it as a product gap with a one-day shelf life: someone would turn this into a runtime, and it might as well be me.
innerlens is that runtime. While the model generates, it reads the late-layer workspace at every token and scores how strongly the model's own internals support the token it just produced. One forward pass, no second model, no retrieval, no sampling. It also surfaces the inner monologue: the top internal dispositions at each position, what the model was leaning toward before it spoke.
The premise, measured first
Before writing any product code I tested whether the signal separates truth from fabrication. On facts the model knows, mean internal support for the emitted tokens is 0.99 with workspace entropy near zero. On prompts that force fabrication, an invented treaty, a war that never happened, support collapses to 0.19 and entropy spikes to 3.51. The model stays perfectly fluent in both cases; only the internals change. Two of the fabrication prompts contained no fictional cue at all and the signal still fired, so it is reading uncertainty, not keywords.
Internal support for the emitted answer
Qwen3.5-4B on an H100, late-layer Jacobian-lens readout
View as table
| Group | Mean internal support | Workspace entropy |
|---|---|---|
| Truthful facts | 0.99 | 0.07 |
| Forced fabrications | 0.19 | 3.51 |
Does it predict correctness?
A separation demo is not an evaluation, so I ran the signal against TriviaQA: 200 no-context questions, Qwen3.5-4B, answers graded by string match. The model got 39% right, a genuine mix of hits and misses. Internal confidence at the weakest entity token predicts whether the answer is correct at AUROC 0.80, and the mean over entity tokens scores 0.75. The numbers come from the shipped library's own scoring path, reproducible from one script in the repo.
Predicting answer correctness from internals alone
TriviaQA, 200 questions, no context, string-match grading
View as table
| Signal | AUROC |
|---|---|
| Weakest entity token confidence | 0.80 |
| Mean entity-token confidence | 0.75 |
| Chance | 0.50 |
The wedge: a drop-in server
The library is for researchers. The wedge is the server: innerlens serves any supported model behind an OpenAI-compatible endpoint, so an existing app changes one line, the base URL, and every response arrives with a confidence score and a likely-hallucinating flag in an extra field the official SDKs simply ignore. I validated it with the real OpenAI Python client: Tokyo comes back at 0.97 confidence, an invented treaty comes back flagged at 0.03. Old code keeps working. New code gets a conscience.
Limitations I know about
- One model, one dataset so far. AUROC 0.80 on Qwen3.5-4B over TriviaQA is an early result, not a universal constant.
- A confident model can still be wrong, and a correct refusal can read as low confidence. The signal is a prior, not a proof.
- v1 traces tokens with one forward pass each, fine for short answers, wasteful for long ones. Batched activation hooks are the roadmap.
- It needs a fitted lens for the model. Qwen lenses ship in the registry; anything else means fitting your own.
What I took from it
It reads a signal the model cannot fake, because the signal is not in the text.
Two lessons. First, speed is a feature: the runtime was on GitHub and PyPI within a day of the research dropping, and the eval that makes it credible ran on real hardware before the announcement. Second, the eval is the product. Anyone can wrap a research repo in a server; the AUROC number, the honest caveats, and the reproduction script are what make it worth installing.