Hello everyone!
I was wondering if anyone has attempted to run any LLMs with an Intel Arc GPU? If so, how were you doing it?
I just bought one of these cards as a cheaper way to get into higher VRAM cards. I've seen a lot of mixed opinions on how useful it is for LLMs at this time. My card will be here in just a few days so I can do some testing, but would be nice if anyone knows where to start.
I currently have an Nvidia 2060 Super 12 GB and it was pretty straightforward to get setup with Ollama.
Thank you!