OpenAI Agent Builder / Widget Actions · Builder’s Console Log 🛠️

OpenAI Agent Builder / Widget Actions

I cannot figure out how to incorporate actions from a widget into my workflow. Apparently it is not possible within the agent builder (?) - so I have also tried finding a solution within Vercel. But I still have had no luck.

Example workflow (inside agent builder)

User: "Write me a summary about my project on 43rd Street"
Agent: *searches vector store*
Agent: *identifies 3 projects on 43rd Street*
Agent: *Displays widget of results* (Each widget has a "select" button)
Agent: "I found three projects on 43rd Street, which would you like me to summarize?"
User: *Presses button from X result inside widget*
Workflow recognizes user's selection, and then writes the summary based on the vector store data from said project

Context:

My widget button carries "onClickAction={{ type: "project.select", handler: "server", payload: { id: p.id } }}"
the widget itself is attached to an agent that responds with search results from the vector store
I am embedding my chatkit through Vercel
I maybe have 12 months of experience learning this industry and I'd argue half of that was spent going in circles with ChatGPT or Claude

My example might not be a valid use-case for a reactive button. But I just don't understand how the agent workflow or any workflow for that matter recognizes a button action / trigger. If anyone can show me this it would mean the world!

8 comments