With user input, you usually know what the threat looks like. With model output, the danger can be a perfectly valid JSON payload that quietly deletes the wrong row, calls the wrong API, or escalates the wrong permission.
One of the biggest mistakes teams make:
They validate that the output looks correct — valid JSON, no profanity, under token limits — and assume it’s safe.
That’s not enough.
What actually matters:
• Validate against the destination system’s contract, not just the response format
• Constrain tool parameters with allowlists wherever possible
• Policy-check actions before execution (OPA, Cedar, etc.)
• Treat model output like untrusted SSRF input — especially when it can touch internal systems
Your LLM is now part of your application’s data flow.
Harden it like any other untrusted system boundary.
Question about LLM output handling? Drop your thoughts below 👇