We accepted a 30% regression for a full day. The fix was already on disk.
Upgraded a 284B local model, measured it 30% slower than the build it replaced, wrote that down as an acceptable trade-off. It wasn't — native multi-token prediction shipped switched off in the new checkpoint. Turning it on: 23.1 → 34.5 tok/s, +49%, and it reverses the regression entirely.
Full breakdown (10 min): https://youtu.be/el8I6LkGh1U
Every number, labelled: https://mtp.novcog.us.com/
The curriculum lesson here is "never compare across versions without an audit" — two of our own acceptance tests were quietly lying at the same time. Full method map: https://hiddenstatedrift.com/method
0
0 comments
Guerin Green
5
We accepted a 30% regression for a full day. The fix was already on disk.
⚡Burstiness and Perplexity⚡
skool.com/burstiness-and-perplexity
AI-native SEO, autonomous agents, and automation pipelines. Built for practitioners who build— not collect. Home of the Hidden State Drift Mastermind.
Powered by