Mike Doolittle

There is a way now. I do completely offline local development with my AgentZero. Granted yes, I do have a 24GB 3090, plus 96GB of RAM, but with that aside you can still do coding with less - yes it IS possible! For example, I ran a coding setup with AgentZero and it only required about 16GB of VRAM and I was getting over 100 tp/s and quality outputs - I did 2 small web apps and the code was flawless! This measures up to the performance of GPT Codex. And I don't pay anything for use! The secret is a really good engine to run your LLM with prefill AND model streaming. I have been waiting for these techniques to be available on private systems for a long time - and now they are here! You CAN run big smart models locally, with speed. So, you can have pro-level coding offline with AgentZero - I am doing it now!

New comment 2d ago

Mike Doolittle

0 likes • 2d

Be specific, state examples. What software, what 3 models are you using and your configurations please.

Mike Doolittle

0 likes • 2d

@Daniel ITwizard Thanks for the reply. I will be glad to work with you. I have used vLLM for main model http://192.168.0.137:8000/v1 (Qwen3.6-35B-A3B-NVFP4) and Ollama nemotron-3.5-lightning:30b-a3b and Qwen 3.8-27b, http://192.168.0.137:11434/v1 (utility model) on the Spark, and the embedding model text-embedding-nomic-embed-text-v1.5 http://host.docker.internal:1234 using LM Studio running on the Mac running A0. I just started to look at strata however when I went back to cloud models, A0 2.14 does noes not run with any models. I was using DeepSeek v4.1 Flash.

Renat Yaraev

Aug 20 •

💬 Discussion

I’m putting A0 on hold for myself, but I won’t stop believing in a better future for it!

Hi everyone! It looks like we’re reaching a point where local development tools are actually improving, rather thanJust seeing the emergence of high-quality agent frameworks like DeepSeek Harness makes it clear why they have become popular. No matter how often I tried using Agent Zero for local development, I invariably hit a dead end. Consequently, this tool will likely remain of little relevance unless its architecture undergoes radical changes—specifically, the removal of the auxiliary ("utility") model; in my view, that is Agent Zero's main problem. Competent memory plugins could surely solve this issue, thereby making A0 more flexible.

New comment 2d ago

Mike Doolittle

0 likes • 2d

@Renat Yaraev I agree with you. I have struggled to use A0 with local models. I have a DGX Spark, one mac with 24g and one mac with 32g to host local models and reached out multiple times for help. You will see several of my posts here. Count the number of times they were answered. The A0 team is moving over to Discord (which does not work for me) which I stated in this Skool group, We have brought up asking for a list of local models to use and the config used is being shared. Little help from any A0 team member. Oh, I did get a comment that local models are difficult, no suggestion from one of the developers of A0. Hermes just works with my local models and has very little issues, unlike A0. I also see many people who have tried A0 and moved on. If the A0 team were interested in A0 being more than a hobbyist tool for people, they would make it a priority to get it solved with suggested models posted on one of their "official" locations. I was told the next version, 2 version ago, should solve the problem. NOPE. I cannot update the Launcher on my mac (which works) yet I can update A0 (which now does not work with cloud models any more.) I suggest you move on to Hermes, or Zo Computer (which can use local models). I also suggest you state any issues over in Reddit AgentZero subreddit if you plan to continue trying to use A0. So why am I still here? I am hoping the A0 team wants a successful harness since they have been working on it for around 2 years. I am also here to let people know that the A0 team does not support local models, which is my opinion. Good luck Renat!

Mike Doolittle

0 likes • 2d

@Daniel ITwizard Can you share with us? I have a DGX Spark, one mac with 24g and one mac with 32g to host local models and I would like to know what models you are using and how you are configuring the 3 models in A0.

Daniel ITwizard

6d •

💬 Discussion

Proof You CAN Run Local LLMs

I see people on here all the time saying that you cannot run AgentZero with Local LLMs. People think that somehow this is impossible - or that you have to sacrifice intelligence - NOT TRUE. I have been running strictly offline local LLMs since day 1 with AgentZero, I pay ZERO api fees! What makes this possible is is having a good engine to prefill the prompt and a good LLM server. Do not use Ollama, do not use vLLM and although I do like it - do not use LM Studio. You need to run a model server with what is called model streaming - this is how you can get around size limitations on your VRAM. I am getting crazy fast speeds with big & intelligent models now, because I take all this a step further and use speed projection, MTP, and the right Quant-size! All of these matter to get quality intelligent output WITH speed. Stop belly-aching that you can't run local LLMs with AgentZero, you can!

New comment 2d ago