I guess I am slow to the party here, but this thread was brought to my attention, so I will try to answer based on what I know about the topic. You can get good results using the built-in Agent Evals workflow when the agent is built on LiveKit Agents. Instead of trying to manually “play calls” over and over, write small behavioral tests (pytest + pytest-asyncio) that exercise a turn or two of conversation and assert what the agent should do — tone, tool usage, error handling, grounding, etc. A few things that have worked well: - Text-only evaluation first (fast + cheap), then only do audio tests when needed - judge() for qualitative checks, e.g., “did it politely greet and offer help?” - Tool-call assertions to confirm the right tool was called with the right args - Mocked tools to force errors and edge cases without touching real systems - Multi-turn tests so we can catch regressions in memory and workflows - Run the tests in CI with LLM keys as secrets, so every PR gets evaluated It’s been great for catching subtle regressions, especially when we tweak prompts or add new capabilities, without breaking older flows. I know many folks I have spoken to also find value in 3rd party tools like: https://getbluejay.ai/ https://hamming.ai/ If you are in the San Francisco area on January 27th and your focus for 2026 is agent reliability, we are having a meetup with industry experts on voice agent reliability techniques and testing. I will share the Luma here in a few days, in case you can join. Another great tool is to use Agent tasks (I've seen some call this templates) and Task groups, particularly if you need to use a specific flow across many customers. You can refine the task and reuse it. You can learn more here https://docs.livekit.io/agents/logic/tasks/ When diagnosing issues, it can be super helpful to use Langfuse or a feature like LiveKit Agents' observability. Here is a nice overview of how and why you would use something like this https://docs.livekit.io/deploy/observability/insights/ (YouTube demo)

For those that can make it and and are interested in voice AI reliability here is our meetup "Voice Mode - The Year of Reliability" on Jan 27th in San Francisco at Frontier Tower https://luma.com/voice-ai-meetup-the-year-of-reliability

Yes there are lots of good ways to get started. LiveKit has an Agent builder where you can design in your browser initially then export the code when you want to start doing much more complex use cases. See here for getting started quickly if you want to go that path https://livekit.io/events/dev-day I am not super clear on your use case but I think a good start could be to connected your agent up to the content of the education program so the users can have an easy way to perform Q&A. I can imagine there will be a strong need for guard rails for a use case like this. Modality will also need to be thought about. Do you plan over telephone, in the browser, voice, text, video, etc. Answers to those will help steer the path to initial implementation. There are a lot of examples from AI enablement platforms. I think a good path forward for you is to find some example project that is a little like what you want to do and use that as your starting point. My personal feel on this is the space is moving very quickly and new and better models are coming around all the time. I would lean towards platforms that give you more flexibility to easily swap out a given model incase they go out of business or something substantially better is released from someone else. In this space there are a few primary paths for getting started. There are more "turn key" solutions which make it pretty quick to get started but may make more complex use cases harder to accomplish in the future. Or there are more flexible frameworks that will require you do to a little more lifting initially but makes it much easer to address future use cases. I am sort of proud of how LiveKit has bridged the gap by making it very easy to get up an running for the new comer as well as being the go to tool for the most advanced realtime AI use cases. Good luck. If you have specific questions I am sure many of us here would be glad to help try and answer.

Looks like you've gotten a lot of good feedback on steps to address your issue. I've never used Zadarma before, but I had a quick look at their website. According to this page, they offer SIP trunk. If that is true, it should be pretty easy to use with LiveKit. I am not sure if your use case is inbound, outbound, or both. Once you have Zadarma trunk configured, you should be able to follow a guide like Twilio's or another provider's, as the steps are basically the same for all of them. Here is a good starting point on the LiveKit side https://docs.livekit.io/sip/ There are lots of inbound and outbound agent examples, too, but I will refrain from sharing them unless you are interested.

@Pablo Arce Mari You can do it self-hosted. One way I have seen folks approach this is to take it in stages to optimise their opportunity costs. You can either build it all at once or build out a piece at a time; it just depends on how much time and resources you have available and time to market risks. Here is one option that I might consider if I were building this: 1. Start by focusing on your self-hosted agent layer and use LiveKit cloud initially 2. Focus on getting SIP running with the new provider and refining your agent layer 3. Once that seems stable, start the LiveKit server self-hosting rollout. Those steps could be a path to production, or at least a path to systematic development. I hope that helps.

The dataset is not open only the weights.

Yes, it is tuned for structured data, including email address collection. Here is a demo from Shayne https://www.youtube.com/watch?v=OZG0oZKctgw