Multilevel RAG AI agent guide

Hey everyone!

I keep running into this problem:

When you have dozens (or hundreds) of files in your vector store, the retrieval quality drops. The more files and chunks you have, the harder it is to get relevant, focused answers - your AI ends up picking from too many candidates, blowing past input limits, and missing the best details. This gets even worse in real-world cases where you need to search through massive, messy data.

If you just do a “classic” vector search across everything, you get a flood of semi-relevant stuff. Answers get diluted. Sometimes the most important facts are lost or buried because your AI’s context window fills up with noise.

To fix this, I created a multi-level RAG workflow in Supabase.

Here’s the idea:

First step: Filter your files using their descriptions/metadata to shortlist only the most relevant ones.
Second step: Run a vector search just within those shortlisted files using their file IDs.
Now you only pull the top relevant results, which are both focused and high quality.

What’s inside the guide/video:

Step-by-step setup for file and document tables (SQL queries included)
How to write better file descriptions for smarter filtering
Code examples for filtering files by metadata and running custom vector queries
How to aggregate file IDs and combine results
Tips for tuning similarity thresholds and managing large file sets
Downloadable workflow template and real use case examples
Results of testing this with real client data (it works much better than a one-step vector search!)

How do you handle retrieval when your RAG projects get big? Are you doing multi-step filtering, pure vector search, or something else?

7 comments

Multilevel RAG AI agent guide