Day 27 – Thinking about secure RAG again and self-hosted LLM

🔥

Today I actually planned to work on something else and i started also to build. But there is one topic I simply cannot ignore anymore. Also the Knowledge I got, but Secure is in the pole position.

If I want to build RAG systems that companies will actually trust, then the system must be secure. The data must stay protected.And ideally, nothing should leave the company environment.

That brought me to one big question:

How do we build RAG with an LLM that does not send data outside?

Someone already commented and suggested hosting an LLM myself. And now I really want to understand this topic.

So here are my questions.

On which cloud oder service is this easiest to do? AWS, Google Cloud, Azure, elestio something else?

Which LLM makes sense for self-hosting?

How would I connect a self-hosted LLM to n8n?Is it via API?

And then the bigger question:

Are there already tutorials, videos or good guides for this? Has anyone seen step-by-step explanations?

I am also curious whether the community plans to show something like this in the future. Because I am sure I am not the only one thinking about it.

A RAG system should not only work well. It also needs to be safe, private and compliant with company expectations.

I really want to understand this topic deep. So if you have experience, ideas or resources, I would love to hear them. I think a lot of people will be looking forward 😁

2 comments