quick cost thing that doesn't get talked about much in automation circles: prompt caching. most client bots send the exact same big chunk every single call. system prompt, business info, FAQ, policies, tool definitions. only the last customer message actually changes. without caching you pay full input price for that whole block every time. with caching the model provider stores that repeated front part, and later calls that start with the same text get billed way cheaper and usually come back faster too. how it works on the two big ones (as i read the docs, double check pricing for your model): Anthropic / Claude: you mark where the cache should end with cache_control. cache reads are billed at a fraction of normal input price, writing to the cache costs a bit more than normal. default cache lives ~5 min and resets every time it's hit, there's also a longer 1h option. OpenAI: it's automatic once the prompt is long enough (1024+ tokens), no code changes. you just get a discount on the cached part. the one rule that matters for both: put the stuff that never changes FIRST and the stuff that changes LAST. if you stick the customer's name or today's date at the top of the system prompt you break the cache on every call and get nothing. where i'd actually use it: - support / FAQ bots with a long knowledge block in the prompt - - agents with a lot of tools defined - - batch jobs running the same instructions over hundreds of rows (lead enrichment, review replies etc) for a busy client bot this can be a real chunk of the monthly API bill, which is nice margin if you're charging a flat retainer. docs: Claude prompt caching: https://docs.claude.com/en/docs/build-with-claude/prompt-caching OpenAI prompt caching: https://platform.openai.com/docs/guides/prompt-caching anyone already doing this on client builds? curious if you've seen how much it actually cut the bill