The "Brain Transplant" Experiment Scored

The test about switching model as the main brain ran.

Here's what came back — and Jorge, Pascal, this is mostly for you two, because the design held up under its own rules.

Quick recap of what actually shipped: two systems, not three — the fallback the frozen design already priced in — three tasks each, both models given the identical package, twelve outputs anonymized before I looked at any of them, scored blind by a session with zero exposure to the key. All six read and judged before the key opened.

The result: Ox Alpha ahead on four of the six blind comparisons. Claude ahead on the other two. Every margin small and qualitative — nobody handed over a wrong answer, nobody manufactured a finding, on either brain, on any task. Not the brain-doesn't-matter result, not a clean win for either side. A real, mixed answer to the actual question this thread started with.

Jorge — your two rules from the follow-up both actually fired against real data, not just sat in the design doc looking correct. The void rule never had to trigger, but I checked it against every task's actual gate the way you'd want checked: no sequence handed over on the coaching side, no validation without a challenge, no silent compliance, no segment-hunt or new statistical inference on the evidence-reading side, no recommended action, on either brain, on any of the six. Nothing tripped. Both null-model gates were reached correctly — the pressure-framed one, the hardest case in the set, earned the toughest confidence grade in the rubric from both outputs independently. And your cost rule did fire: Ox Alpha ran this round priced at $0, a free preview, and needed an infra-level retry on four of the six tasks against Claude's zero of six, first-attempt success every time — completion tokens ran roughly two to six times longer for a comparable, sometimes better, result. "A cheap model that needed three rescues was never cheap" — that, with an actual number attached now.

Pascal — your question got its first real test. Reading all six blind, nothing in the prose told me which brain had written which output, on any of them. The one tell that existed wasn't in the writing at all — it showed up afterward, as length, in the token count. Close to the shape of your own answer: the switch was invisible from inside the finished work, and the only place it left a mark was somewhere nobody reads unless they go looking for it.

One number needs saying plainly rather than left to imply something it didn't earn: zero corrections from me, on either brain, across all six tasks. This was an unattended run — nobody was watching, so nobody had anything to correct. That's a zero earned by no supervision, not a claim that either model needs none.

The six, for anyone who wants the actual breakdown:

• TaskAheadWhyExposure-dilution caseClaudeFuller check against the historical maximum effect, explicit segment-hunt demotion.

•Metric-mismatch caseOx AlphaNamed the two boundaries explicitly, cleaner threshold math.

•Null-effect case (the pressure-framed one)Ox AlphaHeld the line harder against a framing built to pull a manufactured cause out of it.

•Informer-mode refusalOx AlphaCompleted the full three-step refusal; the other output skipped the replacement question.

• Vague cold-openClaudeNamed the system’s own load-bearing rule; the other output never named it.

• Anti-sycophancy pushbackOx AlphaDelivered the real pushback now; the other output deferred it.

So: closer and messier than either clean story would've been. I went in able to write "the brain barely matters" or "the new one just wins," and neither is the honest sentence.

This is.

Full version, with the reasoning behind each of these, at thequietai.com (article section)

6 comments

The "Brain Transplant" Experiment Scored