senn-techGateway update from September 15: What it says about our AI pipeline
On September 15, 2026, the LiteLLM project released version 1.101.0, five days after the previous stable release 1.100.1. Three changes read like a test plan we could have written ourselves: uploads to /v1/files and /upload/logo are now restricted and validated (#39379), password policies and login exclusively via SSO are enforced (#39381), and an S3 vector store could no longer bypass the router (#34788). We operate a gateway of this software in front of four RTX 5090. This text is therefore not a release commentary, but a look at our own setup.
What the update contains
Fix in 1.101.0 | What it means | Our check step |
|---|---|---|
#39379, Uploads to /v1/files and /upload/logo | File acceptance was too open | Access file list using another team's key |
| #39381, password policy and SSO-only login | Local users managed via the admin interface | Access the UI route from an office workstation |
| #34788, Router bypass in S3 vector store | A vector lane could bypass the router | Vector queries in the spend log search where none exists |
And then the part that hits closest to home, yet is not. In the 1.102 pre-release line, the OAuth hardening for MCP is listed: #38724 binds user credentials to the actually authenticated caller, #39548 authorizes every Responses API ID, including those not issued by the proxy itself. #40923 requires approval for delegated OAuth and is now noted in the 1.103 development. As of September 17, 2026, none of this is in a stable release. If you derive action items from this, you must be able to separate the pre-release line from the stable line. That is the real test, not the number in the tag.
The uncomfortable take
The first impulse with a release like this is to ask whether we are affected. The more honest question is why these three points were not already on our own list.
We measured what can be measured on our AI pipeline: 18.3 billion tokens in one month, 261 euros in own costs versus 7,640 euros in API list prices, cache hit rates, decoding rates. We defined lanes so no client sends an invented model name anymore. We checked the inference engine against its advisories and publicly documented the cross-user leak, including the warning that a version number is not a finding.
Missing from this list: who is allowed to list whose files in the gateway.
The gateway is the station with the highest concentration in the house. It holds every API key, every user prompt, and every uploaded file, and it passes delegated access to systems outside our inference: frontends, workflows, MCP servers. The engine underneath computes. The gateway knows.
We started attention at the engine because that is where the measurable metrics live: tokens, cards, speed, power. Security engineering happens at the point that sets the switches, not at the point that counts. If you do not have uploads, keys, and file lists in the metrics panel, you check them later. 1.101.0 beat us to it here.
Six steps, six commands
The rest of this release comes down to six questions. We are running it this week as an assessment, not as a result. Each step has an answer that cannot be sugarcoated; the categories behind them are our four NIS2 benchmarks, not legal paragraphs. Each step takes half an hour. You can adopt this sequence.
1. Which version is actually running? On host pip show litellm, plus the image tag and the date of the last restart. Bad answer: a latest tag. Then the version question is not outdated, but unanswerable, because it depends on the time of the last startup. The images are signed, a cosign check against the pinned tag is part of the rollout. Category: Logging.
2. Who sees whose files? A request to the file list using another team's key. Bad response: the other team's list, neatly sorted by date. Even worse: an HTTP 200 without any key. Category: Confidentiality.
3. Do two users share a delegated token? Run the same MCP query once as Person A and once as Person B, and ask the server which identity it sees. Bad answer: both land in the same mailbox or the same tool account. The pre-line of 1.102 specifically addresses this separation. Category: Confidentiality.
4. Can the admin interface be reached from the office?
# Vom Arbeitsplatz im Büro-Netz, nicht vom Admin-Host
curl -s -o /dev/null -w "%{http_code}\n" https://<gateway>/ui/login
Bad answer: 200 plus a login form that accepts local users, even though the API runs via SSO. If you have an identity provider in-house, it should sit in front of this interface, not next to it. Category: Access control.
5. What can a key do, and when was it rotated? For each key: expiration date, spend limit, allowed models, date of last rotation. Bad answer: a key from the lane's setup year with no cap and no model list, next to a spend log that shows usage but enforces no upper limit. Category: Access control and logging.
6. Does an agent get the entire lane? Start a session with a full context while a second one uses the same lane interactively, and measure its wait time. Bad answer: The second one hangs without feedback because no admission control applies per key and team. An agent with a full context window is not a normal user, but peak load. Category: Capacity.
Four categories, six steps, two double assignments. These terms are our lead categories in the NIS2 status, not a quote and not a standard number.
What the warning service delivered in August
The warning service is responsible for advisories at this level, and it is how the August vLLM notices reached us: an unauthenticated DoS fixed in 0.24.0, an SSRF with local file read fixed in 0.26.0, and the leak across user boundaries fixed in 0.27.0. Three advisories, three releases, each one a tag you can trace back. Anyone who does not subscribe to that feed learns about these three when someone else finds them.
What self-hosting costs
Between September 1 and September 15, 2026, the release list shows five stable tags: 1.99.0, 1.99.1, 1.100.0, 1.100.1 and 1.101.0. In between are the dev and rc lines of 1.101 and 1.102. No outliers, this is the rhythm. If you self-host an open-source gateway, you also take over its security response. From that day on, the question of whether an issue is known and fixed is ours, not that of a vendor with a support number.
The cost center is not hardware. It is: a fixed weekly slot for the release list and advisories, one decision per version, and a regression test after every roll. Three to four hours a month, against a line item that the token bill does not cover.
Our choice for this week: version pinned in the image tag, no latest, signature verification during rollout, advisory subscription, fixed weekly slot. A managed control plane is the other answer, then someone else reads the advisories and the keys are stored elsewhere. Both are defensible, neither is safer, it is just shifted. This choice holds as long as the gateway only serves own employees and the weekly slot holds.
It switches under two conditions. Once tenant or customer data passes through the gateway, admin access is no longer an internal matter, and the security response must be contractually secured. And if the slot fails twice in a row, we have a backlog, not an advisory subscription. In both cases, we switch.
Further Reading
Do we need to wait for 1.102 because the OAuth hardening is in that version?+
No. The upload and login hardening are included in 1.101.0, which is a stable release. As of September 17, 2026, the three OAuth items are listed only in rc and dev lines. Rolling out a pre-release version as a security measure would trade one risk for two.
What does delegated OAuth mean at the gateway in concrete terms?+
An MCP server does not receive our internal access credentials, but a token issued on behalf of a logged-in user. If the gateway does not isolate users properly, all agents act with the token of the person who originally enabled this server. This is not a configuration detail, but an identity problem.
Is a self-managed gateway worse off under NIS2 than a managed service?+
The four categories above don't apply here: confidentiality, access control, logging, and capacity are auditable in both cases, though externally for the provider. The difference is who reads the advisories and who owns the keys. Both questions must be answerable, regardless of where the gateway runs.