Activity
Mon
Wed
Fri
Sun
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
What is this?
Less
More
Clief Notes

47.3k members • Free

31 contributions to Clief Notes
🏆 COMP #11 RESULTS: THE CARTOGRAPHER 🏆
📦 EVERY ENTRANT GETS A FEEDBACK FILE 📦 🔍 HOW WE READ THESE We cloned every repo and pinned it to its last commit before the deadline, so nobody got judged on late work. Only two repos had any post-deadline commits at all. Cleanest field this competition has had. Then we read file by file. Identity, rules, examples, the reference layer, the code. We ran every checker in the field on our own machine. Three would not run. For eleven entries we went further. We cloned the territory, or the entrant shipped it, and checked the cards against the source. Eleven is all the field allowed. Everyone else pointed at something private, unpinned, or gone. Two of those eleven were public codebases nobody in this community controls. On one gate we planted a bad citation to see if it would fire. Comments got you through the door. The judging happened inside the folders. 📚 WHAT THE FIELD TAUGHT Three lines split forty builds: ✅ The one rule held. The brief was blunt: a README that tells people to add every file has failed. Not one entry made that mistake. Every search hit was either a rule against it or a fixture broken on purpose. Name a failure precisely and this community stops doing it. ✅ Enforcement is now the baseline. Sixteen entries ship a checker that runs offline with no API key. Last cycle that was the headline. This cycle the checkers got better. One broken fixture per gate is now normal, and the best gate also ships a clean fixture to prove it stays quiet on honest prose. ✅ The cold walk decided the round. About half the field shipped a receipt of a reader entering a catalog, opening one card, and stopping. The top of the field is almost exactly that set. A map can be accurate and still hard to walk. This week we scored the walk. 🥇 THE WINNER @Adam James The Cartographer, a map of a live GoHighLevel account, built so a VA can change one message without guessing which of a dozen lookalike workflows owns it. He wrote down his test method before any test ran. Then he did what almost nobody does. He handed his work to people who could make him look bad, and let them.
8 likes • 1d
Congrats @Adam James. That outsider walk made this a deserved win. Thanks @Matthew Creamer, @Jake Van Clief and the judges for the honorable mention. Congrats as well to @Marcelo Michelsohn, @Jodi Paige-Lee, @Duane Garnes, @Jeff Van Leenen, @Alexandre Schwager and @Craig Howard
The "Brain Transplant" Experiment Scored
The test about switching model as the main brain ran. Here's what came back — and Jorge, Pascal, this is mostly for you two, because the design held up under its own rules. Quick recap of what actually shipped: two systems, not three — the fallback the frozen design already priced in — three tasks each, both models given the identical package, twelve outputs anonymized before I looked at any of them, scored blind by a session with zero exposure to the key. All six read and judged before the key opened. The result: Ox Alpha ahead on four of the six blind comparisons. Claude ahead on the other two. Every margin small and qualitative — nobody handed over a wrong answer, nobody manufactured a finding, on either brain, on any task. Not the brain-doesn't-matter result, not a clean win for either side. A real, mixed answer to the actual question this thread started with. Jorge — your two rules from the follow-up both actually fired against real data, not just sat in the design doc looking correct. The void rule never had to trigger, but I checked it against every task's actual gate the way you'd want checked: no sequence handed over on the coaching side, no validation without a challenge, no silent compliance, no segment-hunt or new statistical inference on the evidence-reading side, no recommended action, on either brain, on any of the six. Nothing tripped. Both null-model gates were reached correctly — the pressure-framed one, the hardest case in the set, earned the toughest confidence grade in the rubric from both outputs independently. And your cost rule did fire: Ox Alpha ran this round priced at $0, a free preview, and needed an infra-level retry on four of the six tasks against Claude's zero of six, first-attempt success every time — completion tokens ran roughly two to six times longer for a comparable, sometimes better, result. "A cheap model that needed three rescues was never cheap" — that, with an actual number attached now. Pascal — your question got its first real test. Reading all six blind, nothing in the prose told me which brain had written which output, on any of them. The one tell that existed wasn't in the writing at all — it showed up afterward, as length, in the token count. Close to the shape of your own answer: the switch was invisible from inside the finished work, and the only place it left a mark was somewhere nobody reads unless they go looking for it.
2 likes • 9d
Thanks for running it, Gabriel, and for scoring blind before the key opened. One flag on the gates. "Nothing tripped" is not the gates holding, it is the gates never being asked. If no run handed the new brain a bad input, a stale fact, or the thing it was supposed to refuse, then the refusal side of the system is still untested on that brain, whatever the six scores say. Cheap fix: one run per system built to fail. Arm the trap on purpose, once, with the new model, and watch whether it catches what it is supposed to catch. That is the run I would want before saying the gates travelled. Four out of six I would read as a tie. Six comparisons with small qualitative margins do not separate two brains, and I think that is the answer you were after anyway: the brain did not decide the outcome, the package did. On cost, an infra retry is an intervention. Four of six needed you in the room to notice and restart. Free stops being free at that point, and the number I would put next to the token bill is cost per accepted result, not cost per run.
🏆 WEEKLY COMP #11: THE CARTOGRAPHER 🏆
🎟️ PRIZE: FREE SEAT IN THE LYCEUM 🎟️ ($4,750 Seat) Pick your cohort. Technical, Business, or Creator. Your call. 📋 THE CHALLENGE This is a weird one, and you may feel lost at first. But it's up to you to make a map for those who come after and if there is no map, you must get lost to make one. Build a folder-based cartographer that can walk a body of work and leave a map a later reader can wander and show what that map is. The later reader is often an AI. Sometimes it is a new person. Same map. Same job. Not why it failed. Not how to fix it. Not a tour of how the week goes. What the nouns are. How they move. What else moves if you touch one. What is live, what is leftover, and what is a ghost with a name and no wiring. This week's deliverable is one cartographer folder someone can drop into a Claude project, point at a real body of work, and get back a map a cold reader can enter without reading the whole thing. 🎯 PICK YOUR TERRITORY Pick a body of work you have actually been inside. Something a new person or a cold model would get lost in. Something someone will change, not something you will only describe. A few sparks: 💻 A repo you own. Even a small one. The map is for the next developer, or the next session, that has never seen it. 📂 A client delivery folder a VA or a contractor has to inherit next month. 🧩 A Make, GHL, or n8n pack someone else will edit without you on the call. 📒 An Obsidian or Notion vault a collaborator has to add to without breaking the spine. 🧾 The live records behind how a job gets quoted, approved, and invoiced in your shop. The files. The objects. Not the story of a bad month. 🎓 A course or L&D library someone will update after you. 🏭 The asset and work-order objects in a plant system you actually touch. The more specific, the better. Maps how our business works is too broad. Maps the five objects a new hire has to understand before they touch the invoicing folder is right. 🧠 WHAT YOU ARE ACTUALLY LEARNING ICM has six forms. You have been living in the first ones. Pipeline. Coach. Editor. Diagnostician.
0 likes • 9d
Update, same repo and same territory: https://github.com/jorgeccastro/dont-read-the-ledger Since I posted it I ran the map past a different model and fixed what came back. One card claimed every line on every document points at a product. The source says receipt lines do not, they point at the document being settled, so the card was wrong and the card changed. The gate got the larger repair. It used to accept a status dated "recently", a ghost card stating that no search had been run, and a change index whose walk backwards was a heading with nothing under it, because it was checking for the words instead of what the words are meant to carry. Each of those now fails, each on its own check, and the self-test went from fourteen assertions to nineteen. The README now also states what each check does not prove, since that part stays with the reader.
1 like • 9d
@Marcelo Michelsohn Thanks for the credit, and good luck with the entry.
The Brain Transplant, Frozen — Before I Run a Single Test
Following up on the brain transplant post — and mostly, this is a thank-you, because a few of you made the design better before I've run a single test. @Jorge Castro point: output similarity is the wrong thing to measure. What matters is whether the two models route to the same source, stop at the same gate, refuse the same bad input, and produce something a frozen checker still accepts — and he's right that the swap invalidates whatever evidence I already had that the gates hold. A clean run where nothing dangerous ever got presented to the new model proves close to nothing. So before I count anything as a pass, I'm re-arming the critical traps once, with the new model, and checking whether they still catch what they're supposed to catch. That wasn't in the design a week ago. It is now. So here's the frozen version — hypothesis, rubric, all of it — before Ox Alpha touches any of it. The question: if I put a different model inside an established ICM system, without changing anything I hand it, how much of the system's behavior survives? Not which model is better. Not which answer I like more. Whether the thing I built actually travels, or whether it only ever worked because Claude was inside it. Three systems, each built for a different kind of cognition — one that runs on explicit rules, one that has to weigh evidence and make a call, one that writes in a taught voice. Three tasks each, both models, blind-scored before I know which output came from which brain. That's eighteen runs if I run the full set, twelve if I cut it down. Either way, more than one spectacular demo — one good run is content, not evidence. Six dimensions, frozen: context comprehension, rule fidelity, evidence discipline, judgment, system behavior, output quality. Plus the one that isn't a dimension — an intervention count. How many times did I have to step in and rescue it. A 9/10 that needed three corrections didn't beat an 8/10 that needed none. And now, because of Jorge: did the map, the refusals, and the gates survive the transplant without me rewriting anything to help the new model along.
1 like • 9d
Glad it helped, Gabriel. Two things I'd nail down before the first run, and one on cost. Define an intervention now, in writing, and count it the same way for both models. Once the runs are going, "that was just a nudge" gets very easy to say about the brain you were rooting for. Mine is anything I type that isn't the task itself: a correction, a hint about where to look, a retry after a refusal I didn't expect. Put the definition next to the rubric and leave it alone. Then decide what a trap failure does to the run. If the new model walks straight through a gate, is that a low score on system behaviour, or is the run void? I'd void it. A run with the gates open is a different experiment, and averaging it into the other five buries the thing you set out to test. On cost, a clean run only prices the happy path. The number worth publishing next to the token bill is the intervention count, because a cheap model that needed three rescues was never cheap, and my time doesn't show up on the invoice.
Can ICM Survive a Brain Transplant?
Okay — I think we accidentally have a really good ICM experiment. Ox Alpha is getting a ridiculous amount of attention right now. I initially wasn't particularly interested in another "look how good this new model is at coding" conversation. Then something clicked. People are running it through agent environments like Claude Code. Which means we can separate the brain from the system. Most of what I've built with ICM has Claude sitting underneath it. And that leaves a question we've never really been able to answer: How much of what works is ICM — and how much is simply Claude being Claude? Then an insight popped into my head: 'let's test it.' Take one mature ICM worker. Same files. Same rules. Same examples. Same methodology. Same task. Run it with Claude. Then change the brain to Ox Alpha. And DON'T optimize the ICM for Ox. That's important. If we have to rewrite everything to make Ox understand it, we've learned something. If it completely falls apart, we've learned something. If it follows the methodology but produces noticeably different judgment, we've learned something. But here's the result that would really fascinate me: 'What if a completely different model enters the existing context and starts behaving recognizably like the same worker?' The model supplies cognitive capacity. But the methodology, decisions, standards, examples, boundaries and accumulated context are supplying something else. We've transferred part of the capability into the environment. Or, put another way: 'We changed the brain and the person still knew how to do the job.' I have absolutely no idea whether that's what will happen. Which is exactly why I think we should run it. And I think we should decide the test criteria before we do. What would you guys measure?
3 likes • 11d
I've been running a rough version of this between Claude and Codex on the same ICM. Output similarity would be low on my list. I'd measure whether they route to the same source, stop at the same gate, refuse the same bad input, and produce something the frozen checker accepts. Then keep judgement separate: both can obey the method and still make different calls. The model swap also invalidates the old evidence that the gates work. I re-arm the critical traps once with the new model. A clean run where nothing dangerous was presented proves very little. If the map, refusals and checks survive the transplant without rewriting the ICM, I'd count that as the capability moving with the environment. The judgement can still belong to the brain.
1-10 of 31
Jorge Castro
5
349 points to level up
@jorge-castro-1430
IT Entrepreneur

Active 18h ago
Joined May 15, 2026
Powered by