Activity
Mon
Wed
Fri
Sun
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
What is this?
Less
More

Memberships

Clief Notes

29.1k members โ€ข Free

AI Finance Academy

2.1k members โ€ข Free

4 contributions to Clief Notes
๐Ÿงช New benchmark out
New benchmark out of Meta FAIR, Stanford, and Harvard called ProgramBench. The setup: you get a compiled executable plus its docs. Source code stripped. Rebuild the program from scratch in any language you want. Tests check input/output behavior against the original binary. 200 tasks, from small CLI tools up to FFmpeg, SQLite, and the PHP interpreter. ๐Ÿ“Š Results across 9 models: Zero tasks fully solved. Opus 4.7 was the best, passing 95% of tests on only 3% of tasks. GPT 5.4, Gemini 3.1 Pro, and Haiku 4.5 hit 0% in that bucket. The interesting part is section 5. Even the model solutions that "worked" looked nothing like the human reference. Median 1,173 lines vs 3,068 in the original. Flat directories. Fewer functions, each one longer. GPT 5.4 wrote 96% of its final code in a single turn on most tasks and never modified existing files on roughly 40% of runs. ๐ŸŽฏ Why it matters for us: The benchmark separates writing code from designing software. Models can produce syntax all day. They cannot yet decompose a real system into coherent modules, pick the right abstractions, or organize a codebase the way a working engineer would. That gap is what computational orchestration points at. It is also where the durable value lives. ๐Ÿ›  Try it: Pick an easier task from the repo (the paper flags nnn, fzf, gron, and jq as more tractable). Run it against Claude or your model of choice. Watch where you and the model split. Note the design decisions you make that the model never even raises. Post your runs and attempts to create a harness that would allow the model to do it. Wins, failures, weird outputs, all of it. ๐Ÿ“ Paper and Repo: ProgramBench I'm building something on top of this right now. More soon.
0 likes โ€ข 10h
Thanks @Jake Van Clief That is so true. I was building an automation using Claude for my team last week. And inspite of writing a very detailed PRD, Claude got it super wrong. What then I had to do is break the whole process into smaller workflows, create a skill for each and link them up - that gave much better results. You are right, solving very context based workflow is outside of AIโ€™s ability, at least for now. Thanks again for sharing.
From messy to branded profile grid
With very little experience a week ago, I created a project outline to brand out the cover images on Instagram profile grid. In less than 30 min, after going through the foundations i had it completed. Claude code executes with just two words and can make as many cover on demand as i wish and output them neatly as PNG files in a folder. Jake - super thanks! I learned so much in less than a week ๐Ÿ™๐Ÿ™๐Ÿ™๐Ÿ™ See before and after :)
From messy to branded profile grid
1 like โ€ข 2d
Super ๐Ÿ‘
Small Win
Hey Everyone, Just wanted to share a small win for me today. I managed to secure my first sit down with a possible client today. I got talking to our contract manager at work today just general chit chat talking about his role and what it entailed. We got onto how late he has to work and how it was hard on the family so so forth. I said to him what if i could give you that time back through implementing software that did a lot of the heavy lifting when it comes to office admin. he laughed and asked how was that I explained lv learned how to use AI to build systems we went back and forth a bit about AI talking about where it might lead and such and I'm not going to lie basically regurgitated Jakes classes stating things like how its now to have a hand carved rifle will set you back a small fortune but way back when you would of been looked down on, I spoke about the part about human desire and how that will always be there. He was completely taken back to him I was now not just a vet and a carpenter I was the future and Iv been asked to come to the office tomorrow to sit down and talk. I know its only a sit down and nothing confirmed but when i first approached them i was basically laughed out the office and given work via what's app but that small amount of knowledge might be the reason my life changes for the better. So thank you @Jake Van Clief .for giving this course away and helping complete beginners like me a fighter chance of a better future.
0 likes โ€ข 2d
Thatโ€™s the way Sean. So good. Keep it up
Welcome to Clief Notes. Here's where to start.
1. Watch the intro video and introduce yourself in the intro post here 2. Start with The Foundation (free course). Concepts, folder architecture, prompting framework. Everything else builds on this. 3. Check in at the bottom of each lesson. Polls, discussion posts, other members working through the same stuff. Use them. 4. When you're ready to build real things, move to Implementation Playbooks (Level 2). When you're ready to build your own tools, Building Your Stack (Level 3). 5. Post your work. Ask questions. Help others when you can. What are you here to build?
Poll
4917 members have voted
3 likes โ€ข 2d
Thanks Jake and hi everyone, happy to be here. Looking forward to learning and interacting heaps with you guys.
1-4 of 4
Mathew Philio
1
1point to level up
@mathew-philio-5185
Sherwood, Brisbane

Online now
Joined May 7, 2026
INFJ
Powered by