AI & Development

AI & Development2026-10-16· By Franz Senn

Open AI models you can self-host in October 2026: what fits in 24, 128 and 512 gigabytes and what the licence allows

Several open models from the last three weeks can run inside your own walls, with no prompt leaving the company. For a business of 20 to 100 people four things decide: how much memory the weights need, what the licence allows, whether German is supported and whether anyone outside the vendor has confirmed the quality. We read the model cards on Hugging Face, the licence texts, the file sizes and the Hacker News threads of 15 and 16 October. We have not measured any of these models ourselves.

What fits in which memory

Sum of the file sizes on Hugging Face, in gigabytes of 10 to the 9th bytes. Room for the context comes on top. For comparison: an RTX 5090 has 32 GB, a DGX Spark 128 GB, two Mac Studio M5 Ultra together up to 1,024 GB. The 24 GB for Bolt is the model card's figure. (Quelle: File lists of the model repositories on Hugging Face, 16 Oct 2026)

Which models fit which hardware class, going by the file sizes above. (Quelle: Our own summary, as of 16 Oct 2026)

Nemotron 3.5 Bolt: 74 billion parameters in 24 gigabytes

NVIDIA released Nemotron 3.5 Bolt on 15 October, a model with 74 billion parameters, 9 billion of them active, that is meant to fit in 24 GB thanks to 2-bit training. (Quelle: Simple Icons, CC0)

According to the model card Bolt is built for everyday agent tasks and fits in 24 GB of memory with native 2-bit quantization. It has 256,000 tokens of context and reads images and video. German is among the supported languages, next to English, Spanish, French, Italian and Japanese, with English and code as the main languages. The licence is OpenMDW-1.1, which the summary on openmdw.ai describes as having no limit on field of use or company size. It is meant to run under vLLM on A100, H100 or Blackwell.

The vendor benchmarks put Bolt against Qwen3.6-35B-A3B at full precision: GPQA Diamond 86.43 against 83.40 and Terminal-Bench 2.1 50.98 against 44.38. These are NVIDIA's numbers. On Hacker News we found little on Bolt: six points and one comment saying Nemotron is "only good because it's free". That is not enough as a cross-check.

Nemotron 3.5 Super VL: 120 billion for a machine with 128 gigabytes

The larger sibling has 120 billion parameters, 12 billion of them active, and up to 1 million tokens of context. The NVFP4 version comes to about 75 GB in total. NVIDIA names a DGX Spark, one H200 or two H100 as the setup. On GPQA-Diamond the NVFP4 version scores 89.90 against 89.52 at full precision. It does not keep up everywhere: on WebVoyager in the offline test it falls from 77.72 to 66.31, on DeepSWE from 24.65 to 18.2. Those are vendor figures as well.

LFM2.5-24B-A2B: fast, but weaker on knowledge and code

Liquid AI released LFM2.5-24B-A2B on 12 October, 24 billion parameters with 2.45 billion active, 131,000 tokens of context and German among its languages. The Q4 file is 14.5 GB. Liquid states 184 tokens per second on an Apple M5 Max. The model card itself advises against using it alone for knowledge-heavy tasks and recommends retrieval or web search alongside.

The Hacker News thread (287 points) partly confirms that. One user reports it is very fast on a 4090 but "very stupid". Another saw it clearly behind Qwen 35B A3B on general knowledge but ahead of a 9B model at much higher speed. A third gave it a small task in a code project and it failed, where Qwen 3.8 managed the same task.

The licence is the LFM Open License v1.0. We read the text: commercial use is allowed only as long as your company and its affiliates stay under 10 million US dollars in annual revenue. For many businesses of 20 to 100 people that is a real limit.

MiniCPM-V 4.7 and EmbeddingGemma 2: reading images and finding documents

MiniCPM-V 4.7 from OpenBMB (card dated 15 October, trending since mid-October) has 35 billion parameters with 3 billion active and reads images and video. The BF16 version comes to about 70 GB, with variants for GGUF, AWQ, GPTQ and FP8. The licence is Apache 2.0. The vendor claims an advantage over Qwen3.6 35B-A3B at lower compute time, which is a vendor claim.

EmbeddingGemma 2 from Google (card dated 6 October) is an embedding model with 740 million parameters that puts text, images, video and audio into the same vector space, in over 100 languages. The weights are 1.5 GB, the licence is Apache 2.0, and Google names laptops as the target hardware. For document search in a business this is the smallest way in on this list.

Swift and ThinkingCap: Qwen3.8-27B with fewer thinking tokens

Two variants build on Qwen3.8-27B to need fewer thinking tokens. Swift from UkisAI went on Hacker News on 16 September, ThinkingCap from BottleCap AI was announced on 22 September.

Both variants build on Qwen3.8-27B, whose base is under Apache 2.0. (Quelle: Alibaba Cloud / Wikimedia, CC0)

Saving in thinking tokens against the base model according to the model cards. The tests differ, so the values are not directly comparable. (Quelle: Model cards of UkisAI and BottleCap AI, 16 Oct 2026)

Accuracy does not drop equally little everywhere. Swift stays at 88.28 instead of 88.38 percent on GPQA-Diamond but falls from 98.67 to 94.00 on AIME 2026. For ThinkingCap the total consumption on Terminal-Bench fell by only 7.5 percent, Kaitchup writes, because answers and actions do not get shorter. The licence holds both back: Swift is free up to 1 million US dollars annual revenue, ThinkingCap is under PolyForm Small Business and asks larger companies for an agreement.

On our production machine another lean variant runs, Cold-Fusion-GAIN. In our 21-case test series it scored 18 of 21, the same as the base. We have not measured Swift or ThinkingCap.

Muse Spark Open: open, but sized for two Mac Studios

Meta Muse Spark Open has 816 billion parameters, 42 billion of them active, and 1 million tokens of context. (Quelle: Simple Icons, CC0)

Meta released Muse Spark Open on 15 October, but the weights on Hugging Face are only available after approval. We therefore could not read the licence text ourselves. A commenter on Hacker News quotes it: everything is free except a service that gives third parties access to the model when the licensee's total revenue exceeds 20 million US dollars. Then an agreement with Meta is needed. Internal use and features inside your own product are excluded. That is a second-hand quotation.

The smallest Q2 file from ggml-org is about 304 GB, the Q4 file about 470 GB. Meta names two Mac Studios with M5 Ultra and up to 512 GB each. Another commenter points to Artificial Analysis: according to it Muse Spark Open ranks behind Qwen3.8 Flash Next and GLM 5.3 Flash, which run on a single B300. We have not checked that. The card tags only English as a language.

Mistral Large 4 and the models you can rent

Mistral has announced the weights of Large 4 for the end of October. What we know until then is in our post Mistral Large 4 among open-weight models.

If you do not want to self-host, Anthropic's Haiku 5.5 is a new price point: the pricing page lists 0.10 dollars per million input tokens and 0.50 dollars per million output tokens for prompts up to 100,000 tokens, where Haiku 4.5 cost 1 and 5. The new tokenizer produces about 30 percent more tokens, so a task costs roughly 13 percent of the old price by our calculation. Where Anthropic processes the data we have not established.

What we recommend and under which condition

If you start with a single graphics card, try Bolt and the Qwen3.8-27B variants, and first check on 50 real tasks from your business whether German and tool calls are good enough. LFM2.5 fits only if your revenue is under 10 million dollars and a task has to be quick. For document search begin with EmbeddingGemma 2, since it runs on any machine and is Apache 2.0. Nemotron 3.5 Super VL is an option if you have a machine with 128 GB and need to read images. You do not run Muse Spark Open yourself.

Open for all of them: the quality on German business text. We have no independent measurement of that.

How we worked

Read on 16 October 2026: the model cards of Nemotron 3.5 Bolt and Super VL, LFM2.5-24B-A2B, MiniCPM-V 4.7, EmbeddingGemma 2 and Swift on Hugging Face, the LFM licence text, the file lists of the repositories and the Hacker News threads on LFM2.5 and Muse Spark Open. We know the OpenMDW text from the summary on openmdw.ai. The card and the licence text of Muse Spark Open were gated, so the licence details there come from a quoted comment. We have not measured any of the models ourselves, and for MiniCPM-V, Super VL and Bolt we found hardly any independent reports.