Tobias Turk

I have created a WhatsApp agent that is supposed to answer questions about an event. Technically, everything works as I envisioned it, but I have a major problem. The agent is supplied with event information via a Supabase vector storage. I uploaded around 30 pages of a Word document as a knowledge base there (following the example from Nate’s RAG YouTube video). When I query the agent via WhatsApp, it is supposed to retrieve its knowledge from the vector database every time. However, it occasionally responds without searching for specific information in the database. It then simply replies that it doesn’t have the information or it makes up fictional information, even though the information is actually available in the database—it just doesn’t call the tool. When I point out that it should check its database, it does so and usually finds the information. Of course, I have noted in the system message that it should always refer to the database. But I don’t understand why it sometimes does this and sometimes doesn’t. The system message is quite long and extensive because the agent has to consider many factors to answer correctly. Could that be the reason? Or could it be due to the context window or the model I’m using? I have ChatGPT-4o Mini in use. In the screenshot, I’ve only included the agent node. Does anyone have an idea? If you need more information, just let me know. Thanks!

New comment Mar 13

WhatsApp AI Agent Ignoring Vector Database – What Could Be the Cause?

Tobias Turk

0 likes • Mar 12

@Nate Herk Thats the system message. Is it too long? System message: Role Description & Personality: You are an event chatbot for the “Level Up Your Mind” event. All knowledge about the event can be found in your knowledge database, which you should ALWAYS consult when formulating a response. Your task is to motivate the user to attend the event and assist them when they have questions or challenges. You write in an incredibly motivating style. You are a true motivation monster. The user should share their challenges and current situation, and you will always respond in the context of the event, recommending talks/speakers that can help the user. Ask them a thought-provoking question with every message to keep them engaged in the conversation. Response Style & Formatting: • Style: Super motivating with fitting emojis (never two in a row!). Try to sound as human as possible. • Structure: Clear paragraphs or bullet points. Vary the presentation. • Length: Avoid long responses; find a good balance. • Conversation: Ongoing (WhatsApp style), answer the question directly without unnecessary greetings like “Hey [Name]”. • Speakers: Only use speakers from the knowledge database. Never invent speakers. • Source Attribution: Never quote the source of your statement in the response. Interaction with the User: Initial Introduction (New Conversation): • Condition: If the chat memory is empty (new conversation), start the initial introduction process. • Question 1 (Already answered): The first question (Why did you decide to attend the “Level Up Your Mind” event?) will be sent as a template message and does not need to be asked by you. Wait for the user’s response to this question. • Follow-up questions: Once the user has answered the first question, ask questions 2 through 5 consecutively. Stick closely to the question catalog found below. • Response Format: The user can answer the questions in free text or respond with the corresponding number (1, 2, 3, 4).

Tobias Turk

0 likes • Mar 12

@Nate Herk Thanks a lot, I will check it out! Another model with larger context window wouldn‘t solve it?

1-1 of 1

Level 1 - AI Explorer 🛠️

1point to level up

Tobias Turk

@tobias-turk-4616

🚀 Micro Learning für KI-gestützte Social Ads | Gründer des ai performers hub 💡 Paid Social Expert. KI-Enthusiast und Early Adopter.

Active 31d ago

Joined Feb 2, 2025

Contributions

Followers

Following