Warning / Exciting Things For Voice V3

So, its been a long process and a lot of work - literally re-writing our voice orchestration to be native - and we still have a small sprint before we will start flowing traffic that way. Here is just a general guide of somethings that will change, be added and be deprecated.

So, rollout. first - the users using our numbers will start seeing the traffic flow first. Then older numbers. Then trunked numbers. Nothing tangible changes from your perspective besides lower latency.

Tooling -

So, we are adding POST / GET / PUT / PATCH / DELETE to api request types. We also have it where you can configure headers, query parameters, path parameters and body parameters. So, fully customized API calls. We will suppose variables in these as well so you can pass API keys or your access token for GHL. Here's the thing, tools is now running through a proxy of ours that will open up the doors for a couple things:

Tools to run as they do today
Direct API calls (will be a toggle), so instead of being wrapped in a args object with metadata, etc - we will just send the API call. This gets rid of the need for middleware because you can make direct API calls.
Workflow execution; not today or the next week but we are adding native workflows to the platform. You can configure a custom tool to trigger a workflow locally to run multi-agent frameworks and basic automation so we will support a middleware internally.

Abilities -

all stays the same, but we are adding agent teams in voice (chat following right after). So, add different assistants into call to create a team env on the call either through a "prompt change" so we just change the prompt dynamically, or through a team so mutliple voices, etc.

AI Models -

We will be running a custom fine-tuned model thats been fine-tuned for voice orchestration. aka, this llm outputs text-to-speech friendly output to keep weird translations to a minimum. You will be able to select an OpenAI model if you would like and run it with your key if you prefer OpenAI output.

Cloning & Voice Library -

We are making cloning better with a better integration to 11labs, we upped our limit to 40,000 voices, and have an enterprise-dedicated deal with them now for better support, SLAs etc.

Public API & Native Support -

This new infrastructure has been decoupled a ton for full API usage if needed with proper documentation. This also allows us to use it without a GHL connection and with other integrations.

Upload Your Own Background Noise -

We will now support the ability for you to upload and run your own background noise in the call.

----

Deprecating / Needs Attention:

Your trunk may need to be edited to support the inbound webhook url. More on this later.
Deprecating retell_llm_dynamic_variables - this is replaced with a metadata object which will contain assistant_id, contact_id, location_id, to, from, direction, and more. So, if you have flows linking to these, I would consider prepping to use the metadata object instead. We will also send all this data in the headers too if thats needed.

tons of stuff, we are finishing things up and flowing traffic soon.

23 comments