Groq- RateLimitError : tokens per minute · AI Developer Accelerator

Mar 22 • CrewAI

Groq- RateLimitError : tokens per minute

Hi Community,

I've been trying to figure out how to set up my agents to handle Groq rate limits. However not too much luck.

Error during LLM call: litellm.RateLimitError: RateLimitError: GroqException - {"error":{"message":"Rate limit reached for model `mistral-saba-24b` in organization `org_01jm5kkpbkeb7s86t477m0xj9k` service tier `on_demand` on tokens per minute (TPM): Limit 6000, Used 4747, Requested 1691. Please try again in 4.375s. Need more tokens? Upgrade to Dev Tier today at https://console.groq.com/settings/billing","type":"tokens","code":"rate_limit_exceeded"}}

Agent config:

max_iter=5,

max_rpm=28,

respect_context_window=True

Any idea? 🙃

1 comment