User interruptions throwing voice ai agent off track

I'm having crazy issues with one of my voice agents getting thrown off track by user interruptions, causing it to miss key information like names, emails, and phone numbers.

In a typical interaction, the voice agent will start verifying caller info, and the user will say "ok" or "uh huh" or something similar. The agent stops talking immediately, listening for the user speech, then starts talking again, only to be interrupted by the user again. It's very troubling.

Do I fix this with prompt or are there settings in the assistant call settings, to improve this?

12 comments