How would you build an internal AI agent that knows an entire company's knowledge base?

I'm the AI consultant for an Australian mortgage brokerage (Sydney, small team). I want an internal assistant every employee can ask about how the company works — our processes/SOPs, which lenders suit which client scenarios, fees, compliance rules, who does what.

Where I've got to so far:

Anchor use case: internal SOP/process assistant. Not client-facing.
Access: private login-gated web chat portal — staff sign in with their company email, public can't get in.
Guardrail: it must cite its source and say "I don't have that documented — ask X" rather than guess. We're regulated; a confident wrong answer about lender policy is worse than no answer.
The discovery that changed everything: I audited our Google Drive expecting SOPs. There were almost none — it's all marketing material. The real process knowledge is in the director's head. The bottleneck is content, not tech.
My plan: one curated company knowledge-base doc as the agent's brain (small enough to fit in the model's context, so no vector DB), an admin screen to edit it, and logging every question the agent can't answer — that log becomes the prioritised list of what to document next, so it bootstraps itself.

What I'd love input on:

Is "whole KB in the system prompt, skip RAG" sane for a small company KB? At what size does that break and force vector search?
How did you get knowledge out of people's heads? This is my real blocker. Interviews? Recording client calls and mining them? What actually worked?
Anyone shipped this with no-code (Custom GPT / Claude Project / n8n) vs. a custom app? Where's the line where custom becomes worth it?
For a small team, is a login-gated custom app overkill vs. just sharing a Claude Project with team seats?
War stories: what killed adoption of an internal agent? What made staff actually trust it?

Stack: Next.js + Supabase + Vercel, Claude API. Happy to share back what I learn.

8 comments

How would you build an internal AI agent that knows an entire company's knowledge base?