Limits are per-key and tiered by credit added. The tier table lives on the pricing page; this page is about how to handle them in code.
Rate limit headers
Every response carries your current position against the limit.
| Header | Meaning |
|---|---|
x-lcllm-ratelimit-requests-remaining | Requests left in the current window |
x-lcllm-ratelimit-tokens-remaining | Tokens left in the current window |
x-lcllm-ratelimit-reset | Unix timestamp when the window resets |
retry-after | Seconds to wait. Only present on a 429 |
Retrying correctly
Respect retry-after when it is present — it is computed from your actual window, so it is never wrong and never longer than it needs to be. Fall back to exponential backoff with jitter otherwise.
import random, time
import httpx
def call_with_retry(client, payload, attempts=5):
for attempt in range(attempts):
r = client.post("/v1/messages", json=payload)
if r.status_code != 429:
r.raise_for_status()
return r.json()
# Trust retry-after when we are given it.
wait = float(r.headers.get("retry-after", 0)) or (
2**attempt + random.uniform(0, 1) # jitter avoids thundering herds
)
time.sleep(wait)
raise RuntimeError("rate limited after %d attempts" % attempts)Do not retry a 402. Insufficient credit and exceeded budgets do not resolve themselves, and a retry loop against a 402 will simply burn your own CPU until someone tops the balance up. Alert on it instead — see cost monitoring.
Shared vs dedicated capacity
Below the Dedicated tier, your throughput comes from a pool of capacity we contract in advance. Your tier limit is a guaranteed ceiling on what you may request — it is not a reservation of upstream capacity.
- In normal conditions the difference is invisible, and a 429 means you hit your own tier limit.
- During an industry-wide spike, a pooled request may queue behind others. You will see elevated latency before you see errors.
- The Dedicated tier is reserved capacity that is contractually yours. It is a contract, not a checkbox — talk to us.