Billing
Per-key budgets for production and staging
Separate keys with their own rate and daily spend limits keep a runaway test or a leaked key from draining your balance. A setup that works for most teams.
· 3 min read · Usefully team
All of an organization's keys draw on one prepaid balance. That's convenient, and it also means a bug in a staging job can spend money production needed. The fix is to give each key its own limits, so every environment and service has a budget it can't exceed.
The limits available on each key
- Requests per minute: caps how fast a key can send requests.
- Tokens per minute: caps throughput for long prompts and answers, which a request limit alone doesn't catch.
- Daily spend limit: a dollar amount per day. Once reached, the key gets
429 daily_spend_limit_reacheduntil midnight UTC.
Limits are set when you create a key in the dashboard. They apply on top of your organization's own limits, so a key can be stricter than the organization but never looser.
A setup that works
- 1One key per environment and service. For example
web-prod,worker-prod,web-staging,ci-evals. Separate keys also make spend by key readable. - 2Production keys: generous rate limits (they serve users), and a daily spend limit at roughly two to three times a normal day. High enough not to cut off a busy day, low enough to stop a runaway loop.
- 3Staging keys: a tight daily spend limit, a few dollars. Staging rarely needs more, and a stuck test should fail fast.
- 4CI and evaluation keys: a daily limit sized to one full eval run, plus a modest margin.
- 5Personal development keys: small limits, one per developer, revoked when someone leaves.
Sizing a daily limit
Look at spend by key for the last two weeks and take the highest day. For production, set the limit at two to three times that; for everything else, just above it. Revisit after launches or traffic changes. If you're pre-launch, estimate cost before you ship and start there.
Handling the limit in code
A daily spend limit isn't a transient error, so don't retry it in a tight loop. Check the code and degrade gracefully:
import openai
try:
reply = client.chat.completions.create(model=model, messages=messages)
except openai.RateLimitError as err:
code = (err.body or {}).get("code")
if code == "daily_spend_limit_reached":
alert_on_call("Daily AI budget reached for web-prod")
return FALLBACK_MESSAGE # e.g. "Assistant is unavailable right now"
raise # rpm/tpm: let the SDK's retry handle itPair limits with alerts
Per-key limits stop the bleeding; a low-balance alert tells you when the shared balance is getting low, whatever the cause. Set the threshold at a few days of normal spend so there's time to top up before requests start failing with 402 insufficient_credits.
Why this matters for security too
A key with a daily spend limit has a bounded worst case if it leaks. Combined with one key per service, a leak means revoking one key and losing at most one day's limit, rather than an open-ended bill. See rotating API keys without downtime.
Reviewing limits over time
Limits set at launch drift out of date as traffic grows. Put a short review on the calendar every month or two: compare each key's highest recent day with its limit, raise production limits that are getting close, and revoke keys that no longer show any spend. Keys that nobody uses are a risk with no benefit.
Limits and alerts are free to use. Model prices are on the models page, and the pricing page explains how credits are reserved and charged.