@Jake Van Clief Regarding your github for evaluating LLMs on psychometric analysis tests: I'm looking at your attached graph and I have to say that the Variance of the results between a temperature of 1 and a temperature of 2 seem so minute, I'm left wondering what that variable is actually meant to represent, one. And, two: its effect is definitely well within the range of 1 standard deviation, so much so we might have plot a new section on a VAR graph to describe it... we'll call it: (σ ^0), that's 0 standard deviations for anyone that doesn't speak math. (If you found this funny, you're a genius and I salute you). But seriously Jake, I'd like to know what this Temperature variable is actually doing to the math and also did you try setting it to 3, or 4, or 5. What does that produce? Also to trust the results you need at least 180 occurrences to have enough data for the law of large numbers to apply; I know for sure I've tested it in real life many times. If that's difficult to compute, you can get by with 150, which will give you a 5% margin of error, which is too much for my liking, but in my experience anything less is just not reliable. So, serious questions here about the data set if you were looking to make it airtight. Not that the test is wrong, you just have to maybe run it more times, as your repetitions per question is set to 20, but I'm not sure if that actually represents the number of occurrences in the data set. Does it? If not, how many times did you run it? Thoughts? *My post here is not ai, I studied financial math and statistics when I worked at TD and Schwab in Margin-Risk Management, plus I built the financial model at my last firm. Statistical mathematics is an area of particular interest to me. Also, not too many people to talk to about data sets... but I find them very fascinating. So I hope you aren't taking me the wrong way here and I appreciate everything you're doing.