AI Local setup - news of this week.

Interesting AI news for anyone following local AI and personal knowledge systems. A new proof of concept called Colibrì can run a frontier-scale 1.5 TB MoE model using just 25 GB of RAM by streaming experts from an SSD on demand. *No high-end NVIDIA GPU required*. It's far too slow for practical use today, but the architecture is interesting and could eventually make much larger local AI assistants possible.

Here is a link to an article about it

https://www.tomshardware.com/tech-industry/artificial-intelligence/colibri-proof-of-concept-gains-frontier-level-1-5-tb-ai-model-novel-approach-runs-on-only-25gb-of-ram-and-shows-promise-for-local-ai-setups

5 comments

AI Local setup - news of this week.