Self-Hosted LiteLLM Gateway: Two Lanes Instead of a Wildcard

Our local LiteLLM gateway was consolidated on 19 August. There is no longer a wildcard fallback lane; instead there are only explicit routes: thinking/nothink on the 4× RTX 5090 host and hermes on the single-card box. Unknown model names now fail loudly instead of being silently rerouted.

Two lanes instead of one wildcard

Vision aliases were also moved to the .3 host, so image processing and text generation sit on the same GPU group.

MCPs and hooks

Behind the gateway sit the local Model Context Providers: docling, excel, erp_lookup, browser_eyes, grounded_docs, searxng, weather, and crawl4ai. Two hooks sit inline: a guard_hook for content checks and an OCR hook for documents.

What actually changed

• Wildcard removed: No * fallback; unknown model names now return an error.
• Vision consolidated: Image endpoints point to .3 instead of scattered cards.
• Explicit lanes: thinking/nothink and hermes are split technically and semantically.
• Hooks active: guard_hook and the OCR hook run on every pass-through.

Takeaway

A wildcard gateway is convenient but opaque. Consolidation makes behavior predictable: every client request must deliberately fit a lane. That adds configuration work, but it prevents silently serving the wrong model.

Further reading

• LiteLLM proxy docs
• LiteLLM on GitHub
• LiteLLM router & load balancing
• vLLM documentation
• Qwen blog