Following on from Part 1 of this series on AGI which set out the need for an AI systems's ability to build comprehensive abstractions, this post reviews different levels of abstraction and how the strengths of LLMs and symbolic AI could be combined into even more capable systems than we have today, that are not only amazing at approximating but also capable of precise, step-by-step reasoning. https://open.substack.com/pub/bockster/p/abstracting-fast-and-slow?r=ionsk&utm_campaign=post&utm_medium=web

You probably have seen / read about the control centres that direct self-driving cars in critical situations. What surprised me that in some cases this number is 1.5 ppl per car. more than if these taxis has drivers. Maybe its just the cost of advancing technology, but its certainly a good illustration of some of the problems that continue to plague AI technology: dealing with out-of-distribution events. https://www.nytimes.com/interactive/2024/09/03/technology/zoox-self-driving-cars-remote-control.html PS: is it fair to put this under 'How it works'? 😅

GPT-4 passed the Bar Exam and the SAT, and AI models crush benchmarks with every new release. All, but one: on ARC-AGI you would score 5-10x as well as your favourite AI model. You can actually try it and might be surprise how current AI models could struggle with it as much as they do. To see how ARC-AGI is different and find out why there is a US$1m prize on cracking it, head to Guaranteed Dissent! https://open.substack.com/pub/bockster/p/arc-agi-one-to-rule-them-all?r=ionsk&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Visual explanation of how GPT-2 (transformer based decoder network) generates the next word https://poloclub.github.io/transformer-explainer/ Credits: Georgia Institute of Technology.

Came across this interactive Transformer Architecture explainer. You can enter a sentence and follow each token through the matrix, self attention, QKV multiplication etc all the way to the end with probabilities for the next token. https://poloclub.github.io/transformer-explainer/