Cost monitoring & alerts

The failure mode worth engineering against is not a slow month. It is an agent loop that runs unattended overnight. Caps are the defence; alerts are how you find out before the cap fires.

Per-request cost

Every response tells you what it cost, so you never have to model pricing yourself or wait for an invoice to find out.

resp = client.messages.with_raw_response.create(...)

cost = float(resp.headers["x-lcllm-cost-usd"])
saved = float(resp.headers["x-lcllm-list-cost-usd"]) - cost

logger.info("llm_call", extra={
    "request_id": resp.headers["x-lcllm-request-id"],
    "cost_usd": cost,
    "saved_usd": saved,
})

Budget caps

Set a hard monthly ceiling per key. Past it, requests return 402 lcllm_budget_exceeded and stop costing you money. The cap is enforced by us before the request is forwarded, so an exceeded budget cannot spend a cent upstream.

• Give every non-production key a cap. Staging does not need a blank cheque.
• Size the cap to what a runaway loop could plausibly burn in an hour, not to your monthly budget.
• Caps are per-key, so one key hitting its ceiling never takes production down with it.

Webhook alerts

Register a webhook and we will post to it on threshold crossings, so the first thing you hear is a warning rather than a 402.

curl https://api.lowcostllm.com/v1/alerts \
  -H "Authorization: Bearer $LOWCOSTLLM_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/hooks/lcllm",
    "events": ["balance.low", "budget.threshold", "spend.spike"],
    "balance_threshold_usd": 100,
    "budget_threshold_pct": 80,
    "spike_multiplier": 3
  }'

• balance.low: account credit fell below your threshold.
• budget.threshold: a key crossed a percentage of its cap.
• spend.spike: the current hour is running at N× the trailing 7-day average for that key. This is the one that catches runaway loops.