Skip to content
senn-techsenn-tech
Back to blog
KI & Entwicklung
AI in production2026-11-22· By Franz Senn

Token budget per role: LiteLLM keys and teams tested

Agents burn tokens in runs that nobody approves one by one, and at month end someone asks which role it was. In a blog post of 28 August 2026, IDC reports two figures from its FERS survey (wave 4, July 2026): 67 percent of enterprises ran over their agent spend budget by more than 10 percent within twelve months, and only 45.4 percent have real-time dashboards for token consumption and cost per workflow. The page gives no sample size. On 22 November 2026 we measured in a throwaway environment what LiteLLM actually does at this point.

LiteLLM wordmark
LiteLLM is a gateway with virtual keys, teams and budgets, documented on its proxy docs pages. (Quelle: LiteLLM / Wikimedia Commons, Public domain (PD-textlogo), trademark: LiteLLM)

A token budget per role caps how much usage a group of accesses may cause in a period, converted into money. LiteLLM has no object type called role. You model the role as a team, attach one virtual key per consumer and set max_budget and budget_duration. Once the limit is reached, the proxy rejects the request with HTTP 422 by default, before it reaches the model provider.

What to make of the figures in the press

Two other IDC figures circulate online, 72 percent budget overrun and 14 percent cost attribution per agent. They were not on the IDC pages we opened, so we cite only the two above, which we read.

How LiteLLM stores budgets

In LiteLLM a budget hangs on an object: key, team, tag, customer or user. The documentation (vendor, retrieved 22 November 2026) describes the fields as follows.

Path of a request through the budget checksRequest with virtualkeyAuthorization header,optional tags in the bodyCheck keymax_budget, tpm_limit,rpm_limitCheck teamTeam budget covers all keysof the teamCheck tagTag budget via metadata.tagsModel provideronly if every check passed
(Quelle: LiteLLM docs, proxy pages Users, Team Budgets, Tag Budgets)
Documentation statementContent
max_budgetDefault null, then nothing is checked
budget_durationFormat 30s, 30m, 30h, 30d; never reset if not set
Reset intervalThe server checks every 10 minutes by default
30dReset on the 1st of the month, 00:00 UTC
Error status422 budget_exceeded, switchable with budget_exceeded_status_code: 429
Models without costBudget check is skipped when input and output price are both 0
tpm_limit, rpm_limitTokens and requests per minute, per key, user or team

Measurement carrier: vendor documentation, retrieved 22 November 2026, units as stated per row. We did not re-measure every row here; the rows with our own result are in the next table.

What we measured

Setup: LiteLLM 1.110.1 from PyPI (not the poisoned version 1.107.0, see LiteLLM 1.107.0: malicious code and key rotation), PostgreSQL 16.15, a small Python server as a stand-in model, one Linux machine with an ARM processor. Every request returned 10 input and 10 output tokens. The test price was 0.001 per token, so 0.02 per request, and budgets were 0.05. Docker was not available on the machine, so the stack ran as a Python environment without containers. Everything stayed local, no production keys were used, and the environment was deleted afterwards.

Test (own measurement, 22 Nov 2026)SettingResult
Key exceeds budgetmax_budget 0.05Requests 1 to 3: 200, from request 4: 422 budget_exceeded, message reports 0.06 against 0.05
Fast sequence10 requests, 0.15 s apart3 times 200, then 7 times 422
Resetbudget_duration 30s, check interval set to 5 to 6 sblocked after 3 requests, first 200 after about 21 s, budget_reset_at moved forward by 30 s
Team budget against keyTeam 0.05, key A with 1.0, key B noneA, B, A get 200, then the team blocks both keys, team spend 0.06
Key stricter than teamTeam 1.0, key 0.05Key blocked after 3 requests
Tag budget/tag/new 0.05, tag via metadata.tags in the body3 times 200, then 422 Tag=role-support
Request limitrpm_limit 2third request 429 throttling_error, retry-after: 60
Token limittpm_limit 30, 20 tokens per requestfirst and second request 200, third 429, type tokens
Model with price 0max_budget 0.058 of 8 requests 200, spend 0.0
Model without a pricemax_budget 0.058 of 8 requests 200, spend 0.0

Measurement carrier: LiteLLM 1.110.1, Python 3.14, PostgreSQL 16.15, Ubuntu on an ARM VM, stand-in model. For the test the reset check was lowered to 5 to 6 seconds. The documentation says 10 minutes by default, and we did not measure that delay.

Four findings

No price, no budget. This matters most for operators of their own GPU lanes: a model whose cost LiteLLM does not know consumes zero budget. The documentation says so for prices explicitly set to 0, and we additionally measured it for a model with no price at all. For self-hosted models you have to enter an internal charge-back price in input_cost_per_token and output_cost_per_token. It is not a market price, it is the unit in which roles are compared. For cost and power draw per token see the comparison price and watts per token.

One request of overshoot. The check runs before the request against the spend so far. The third request started at 0.04 and ended at 0.06, then the key was blocked. For agents with long answers that one request can be expensive, so a budget is a cap plus one request.

Both levels apply. Team budget and key budget were checked independently in our test, and the tighter limit won. The documentation says the owner's personal budget does not apply to team keys. That is a different level from the budget on the key itself, even though the wording blurs the two. For planning: the team budget is the roof of the role, the key budget is the limit per agent or service.

Parallel requests on a fresh key. Five runs with ten simultaneous requests on a new key gave one 200 and nine 422 every time. In the four runs where we kept the error text, the message reported a current spend of 65.6 and 98.4 for a key that had spent 0.02. We did not investigate the cause. Anyone starting a service that sends bursts on new keys should reproduce this first.

Two limits affect reporting. soft_budget is accepted when a key is created but not stored (/key/info showed null), and the documentation does not describe the field for keys. Setting tags directly on a key was rejected with 403; according to the proxy message the feature is part of LiteLLM Enterprise. Tag budgets via metadata.tags in the request body worked without a license. On accounting we saw this: the counters on key and team were correct after 12 to 25 seconds, but /spend/tags listed only the user-agent tags, and the table of individual requests held one of 36 successful requests (counted from our test sequence). We therefore did not reproduce reports by tag or by request, only the spend counters.

How to set up roles

The steps are a recommendation from the test, not a description of our operations:

  1. One team per role with max_budget and budget_duration: "30d", so the counter returns on the first of the month. We saw that date on the tag budget: budget_reset_at read 1 December, 00:00 UTC.
  2. One key per agent or service inside the team, with its own smaller limit and rpm_limit as a brake against loops.
  3. Enter a charge-back price per model before the first budget. Then send three requests with a test key and check the spend.
  4. Prepare clients for 422, or switch the status to 429 so retry logic does not run endlessly.

For per-customer budgets see LiteLLM customer management without a stable release, and for token cost per task see the agent harness study. More posts are collected under the topic AI in production. If you need a gateway with roles, budgets and key management for your own company, the frame is described at Corporate LLM.

Where our gateway stood

According to our documentation of 6 August 2026, the gateway of that time handled 14,435 requests in seven days under a single master key, without per-consumer virtual keys. We did not check the current state for this post, and we do not claim that we use max_budget. How that gateway is built is described in LiteLLM gateway in self-hosted operation.

If you run a gateway with one shared key, set the charge-back price per model first, then a team budget per role and a key per consumer, and confirm with three test requests that the spend actually rises.

Further reading

Questions?
Which HTTP status does LiteLLM return when a key budget is used up?+

422 with the error type budget_exceeded. We measured this on LiteLLM 1.110.1 on 22 November 2026 for key, team and tag budgets. The documentation names 422 as the default and describes budget_exceeded_status_code: 429 under litellm_settings for clients that expect 429. We did not test that setting.

Do budgets work for self-hosted models that have no price?+

No. In our test the spend stayed at 0.0 after eight requests against a budget of 0.05. This held for a model with an explicit price of 0 and for a model with no price at all. Anyone running vLLM lanes behind LiteLLM has to set input_cost_per_token and output_cost_per_token per model, otherwise every budget has no effect.

For a key inside a team, does the key budget or the team budget apply?+

Both, and the stricter limit wins. In our test a team budget of 0.05 blocked even a key with its own budget of 1.0. Conversely a key budget of 0.05 blocked the key in a team with a budget of 1.0. The documentation adds that the personal budget of the key owner does not apply to team keys.