Hey everyone — been building local-first AI infrastructure and this community is exactly my vibe. I run a full AI pipeline from a single laptop (RTX 5080 16GB, 32GB DDR5) — Ollama in Docker with GPU passthrough, PostgreSQL, Redis. The philosophy: 80% of AI workload runs on free local models, only the 20% that needs frontier reasoning hits a cloud API. Cost per pipeline run dropped from $8-15 to $0.15-0.40. I've shipped a few tools with this setup — market scanners, a knowledge retention engine with RAG, and a live SaaS API product. All from the same machine. What brought me here: I want to add a voice layer. Seeing folks run Pipecat with local STT/TTS on consumer GPUs is exactly the direction I'm heading. My Ollama stack already handles LLM inference — pairing that with local Whisper or the new NVIDIA Nemotron STT model on the same GPU seems like the natural next step. A few things from the recent threads caught my eye: - @Kwindla's sub-500ms voice-to-voice on an RTX 5090 with Nemotron — curious how that scales down to a 5080 with 16GB VRAM when the LLM is also loaded - @Jin Park's custom orchestration engine replacing Vapi/Retell — that modular approach maps directly to how I route pipeline stages between local and cloud models - The latency discussion around local vs cloud STT — has anyone benchmarked Whisper locally against Deepgram for voice agent round-trip times? Looking forward to learning from this group and sharing what I build.

I'm currently handling about 6k calls per day between a few different enterprises. We're likely going to implement and build this all out in LiveKit but I also need a better eval + simulation solution and I don't really want to build it out as well. I have seen Coval but I don't know the cost yet -- what else it out there that you guys use that can give me actionable feedback of missed tool calls, hallucinations, etc.

I’ve been building voice agents primarily with Retell and VAPI so far. I’ve explored platforms like Bland and Bolna as well, but I’ve mostly stuck with Retell since it’s been solid and reliable. Recently, I wanted to branch out into custom-built voice agents using frameworks like Pipecat, Cartesia, and others. I’m not really a coder, but I started experimenting with the Cartesia Line SDK through Antigravity. I managed to put together a decent agent but running into a lot of errors as I'm trying to match the responsiveness I get with Retell. No luck yet. I came across this community through a friend and thought I’d ask: has anyone here tried building with the Cartesia Line SDK? I’d love to hear your experiences.

Hey Guys! I'm fairly experienced with VAPI, but 100% new to open source voice AI. I'm currently looking to get deeper into that, and wondering what the state of the art/commonly used providers/models for STT, TTS, LLM, turntaking?, ... are, that are open source/hosted/usable in the EU. I had an advisory call today with a start-up that wants to integrate voice AI, and since they're in GER they need to be compliant and I want to recommend them a few good and solid options for their infra. Thank you already for letting me know your thoughts! Happy to learn! Cheers!

Hi all, I am Eric, COO at Cloudonix a Multi-Award Winning AI Voice Agent Orchestration Platform - Humans and AI Speak as One. Cloudonix is redefining how agentic AI voice agents connect to the real world. Our API-first, telecom-grade platform makes it faster and easier to deploy, scale, and operate voice agents—without complex infrastructure or specialized telecom engineering.