GPU Sizing for LLMs · AI Developer Accelerator

GPU Sizing for LLMs

II have seen quite a few people asking about GPU sizing for running LLM's recently.

I just found the following article on Medium, which lays out a formula for calculating such things. However, this article failed to cite the origin, which is here.

Also, in the original article, I found a link to an excellent program: "Can you run it? LLM Version," hosted on Hugging Face. The article and the tool are worth a look, even if it is just for curiosity's sake.

And just because it's the weekend, here is some Transformer Math 101 in case you haven't seen it. I hadn't. There may be a test later :P

3 comments