Feb 16 • 💬 General
Andrey Karpathy's last video about LLM
Andrey Karpathy's last video about LLM is great, especially for the beginners of the topic. Here are the chapters of the video:
- 00:01:00 pretraining data (internet)
- 00:07:47 tokenization
- 00:14:27 neural network I/O
- 00:20:11 neural network internals
- 00:26:01 inference
- 00:31:09 GPT-2: training and inference
- 00:42:52 Llama 3.1 base model inference
- 00:59:23 pretraining to post-training
- 01:01:06 post-training data (conversations)
- 01:20:32 hallucinations, tool use, knowledge/working memory
- 01:41:46 knowledge of self
- 01:46:56 models need tokens to think
- 02:01:11 tokenization revisited: models struggle with spelling
- 02:04:53 jagged intelligence
- 02:07:28 supervised finetuning to reinforcement learning
- 02:14:42 reinforcement learning
- 02:27:47 DeepSeek-R1
- 02:42:07 AlphaGo
- 02:48:26 reinforcement learning from human feedback (RLHF)
- 03:09:39 preview of things to come
- 03:15:15 keeping track of LLMs
- 03:18:34 where to find LLMs
- 03:21:46 grand summary
4
0 comments
Andrey Karpathy's last video about LLM
skool.com/data-alchemy
Your Community to Master the Fundamentals of Working with Data and AI — by Datalumina®
Leaderboard (30-day)
1
+203
2
+66
3
+17
4
+12
5
+12
Powered by