OpenAI Real-time conversation with AI / N8N

I had a client that wanted to train sales people in real-time with different personas, and then evaluate their performance. I really thought about this for awhile, and knew it was possible, but expected it to be complicated. Turns out, the base functionality in n8n can be accomplished with just 3-4 nodes - way simpler than I initially thought.

The breakthrough for me was the "generate HTML" node. This lets me call my webhook directly from any web app or link, which is pretty slick. In this setup, I configure a personality for the AI before doing the OpenAI request/Whisper:

json{

"instructions": "{{ $json.ai_instructions }}",

"model": "gpt-4o-realtime-preview-2024-12-17",

"modalities": ["audio", "text"],

"voice": "alloy",

"input_audio_transcription": {

"model": "whisper-1"

}

Here's the cool part - the HTML template gets returned to the user, building the page on the fly. So the agent on the other end takes on the specified personality (like a roof salesman), and the conversation feels surprisingly real. When you wrap up, the AI evaluates the transcript and scores your performance based on your criteria.

Some other interesting possibilities that came to mind:

Interview preparation with AI playing tough interviewers

Negotiation training with different difficulty levels

Customer complaint handling with progressively angrier customers

Medical students practicing patient consultations

Language learning through realistic conversation scenarios

Security training for social engineering attempts

This approach cuts down development time dramatically while still delivering a powerful training tool. The applications are pretty much endless, and it's all easier than expected.

Best,

Brian

6 comments