The bottleneck sits in system memory: what one RAM test reveals about local AI agents
AMD showed the Threadripper Halo Station at IFA: a workstation for local models with up to 576 GB of graphics memory (with four cards) and up to two terabytes of system memory. AMD gives no date according to TechSpot, itdaily writes of a 2027 market launch; TechSpot counts 576 GB with four cards, itdaily with two (IFA report, spec overview). At first glance that reads like bragging. Anyone who works with local models knows the card is what counts. The tech channel Alex Ziskind turned exactly this assumption around in a video from November 5: by his account he swapped the memory for this test: out went 128 GB, in came 256 GB, same model. The video description also mentions that he pulled two RTX PRO 6000 cards; whether the rest of the test setup stayed the same we did not verify. The number that sticks: one run with twelve coding agents took 85 minutes before and brought two of twelve jobs across the line. After the upgrade: 59 minutes, twelve of twelve. The runs end differently, so the minutes do not compare equal amounts of work. Every video figure in this post is reported by the tester (status: reported); we had no transcript.
What a parked context is, briefly: A parked context means the conversation history of an AI agent sits in the machine's system memory between two compute steps, instead of being pushed into the model again at every step. An agent waiting for its next tool answer occupies RAM, yet no GPU time. How many agents a machine can serve at once therefore hangs on two places: the memory slots of the GPU and the RAM size of the machine. Which one fills first is a question to be measured.
What was measured
The tester let a growing number of real coding agents run against one local model and counted how many jobs finish inside a fixed deadline. Between conversation rounds the contexts sat parked in system memory. The results per the video: eight agents on a 32 GB R9700 setup: 7 tokens per second on average, all finished. Twelve and sixteen agents with 128 GB RAM: a parked context got evicted from memory 30 times, ten of twelve jobs failed, and the run took 85 minutes. After the swap to 256 GB: zero evictions, all twelve finished, 59 minutes. RAM use peaked at about 150 GB, by our reading on the 256 GB configuration, since that figure is physically out of reach with 128 GB; we did not check the video frame by frame for it. Speed per token stayed unchanged at roughly 7 tokens per second. The machine showed no swap activity the whole time, and the CPU sat at about 6 percent load. Eviction and rebuild are the obvious explanation; it is not proven, because the graphics cards changed in the same upgrade.
A second data point from the same video fits here: on a 32 GB pro card the tester ran a mixture-of-experts model with 124 billion parameters, Ling 3.0 Flash from InclusionAI, which activates only about 5 billion parameters per token. Just over 5 GB of it sat on the card, about 90 GB in system memory, and output speed held at 38 tokens per second. Anyone who wants to run such a model on a normal card inventory needs, going by this single data point, RAM channels and RAM volume first, and a second compute card is the wrong lever for it.
The math against our own fleet
Numbers from one video are a hint. So the question: what does memory look like here? Measured on November 15, 2026 over SSH, read-only, on two hosts: free -b and /proc/meminfo for the host, docker stats --no-stream for the largest container, dmidecode -t memory for the population. Unit GiB.
| Host and role | VRAM | RAM total | Largest container |
|---|---|---|---|
| Inference host, 4 × RTX 5090, vLLM | 4 × 32 GB | 91.8 | 86.0 |
| Gateway host, LiteLLM router | none | 26.9 | 1.1 |
On the inference host a single vLLM container occupies 86.0 of 91.8 GiB, and free reports only about 1.3 GiB as available. Two caveats: docker stats counts page cache and shared memory, and the host reports 64.9 GiB as Shmem. We did not break that share down. What is established: memory is tight on this machine and not on the gateway. An earlier measurement on November 7 put the same container at 82.8 GiB, so the value moves. The population: six 16 GB modules, one per channel according to dmidecode, configured at 4,800 MT/s against a module rating of 5,600 MT/s (EPYC 8024P, six memory channels to our knowledge, not looked up again at AMD).
What this means for your own machine
Our order when buying capacity for agent operation therefore: first check whether the model plus preprocessing fits into RAM, then check how many sessions should run at once, and only at the very end think about the next card. A second, often overlooked point is the width of the memory connection. On our inference machine every channel holds one module, so there is nothing to add there. Anyone planning local AI in operation should ask for the RAM population and channel count explicitly in the quote.
The method is open for rebuilding: the tester published his benchmark as Agent Headroom on GitHub (Apache-2.0, created October 31, 2026), five tests, one of them for agent contexts kept warm in host memory. On November 15, 2026 the project stands at 19 stars and version 0.1.0 per its README: a usable rough build, nothing more. Yet it measures exactly the quantity missing from every product page: how much agent work actually passes through your machine between GPU slots and system memory. If you want your own capacity measured with us, see corporate AI as a consulting service: model choice, memory planning, commissioning, operation.
The topic page collects further measurements on the capacity of local AI: Local AI and GPU hardware.
RAM channels first, the next card later
AMD advertises two terabytes of system memory for the Halo Station (itdaily, market launch 2027; TechSpot lists 2 TB as the processor maximum). The question of the local AI machine can no longer be answered by card size alone. The measurement from the video is simply built and the direction is plausible; the only thing we checked ourselves is our own RAM situation: system memory decides which and how many agents run, the GPU decides the pace per token. At purchase time that means: RAM budget and memory channels before the next card, and before any upgrade one own measurement of what the load really occupies.
Open points: We could not verify the video figures (85 and 59 minutes, 30 evictions, 150 GB peak, 7 and 38 tokens per second, 90 GB in RAM): no transcript, and the video description does not list them. Whether the graphics cards changed between runs is open. For Ling 3.0 Flash the model card confirms 124 billion parameters in total and 5.1 billion active, not the split between card and RAM. The composition of the 64.9 GiB Shmem on the inference host and the CPU's channel count per AMD are unchecked. The date of the TechSpot report was not readable from here.
Read next: NVIDIA snapshot: restore versus cold start · AMD EPYC 9006 Venice with 256 cores · running Nod from n8n Labs ourselves · DSpine as a draft model for Qwen3.8-27B
Further Reading
Status per source: verified = fetched and read by us on 15 November 2026, reported = statement by third parties, derived = derived by us.
- Alex Ziskind, video of 5 November 2026 – reported: measurement with 128 and 256 GB of RAM; title, date and chapters verified, the figures themselves not
- Agent Headroom on GitHub – verified: Apache-2.0, version 0.1.0, five tests, 19 stars on 15 November 2026, created 31 October 2026
- Ling 3.0 Flash on Hugging Face – verified: 124 billion parameters total, 5.1 billion active, MIT licence
- TechSpot on the Halo Station – verified (read through a browser tool, direct request returns 403): 576 GB HBM3E with four cards, up to 2 TB DDR5 on the processor, no date
- itdaily on the Halo Station – verified: up to 2 TB RDIMM, market launch 2027, 576 GB with two cards
- Own measurement 15 November 2026, derived:
free -b,/proc/meminfo,docker stats --no-stream,dmidecode -t memoryon the inference and gateway hosts, read-only
Does a company LLM need 256 GB of RAM?+
No. That figure belongs to an experiment with many parallel coding agents. Normal operation follows another order: count the weights first, add the overhead for context and preprocessing, then round up. We give no fixed rule of thumb: on our own inference host a single vLLM container occupies 86 of 91.8 GiB (page cache included), although the model fits on the cards. It gets tight with many concurrent sessions or offloaded weights.
Why did the agents behave so differently with more RAM?+
Per the video, the old setup evicted the agents' parked conversations from system memory, and every evicted conversation had to be rebuilt completely. More RAM prevents that back and forth. Speed per token stayed the same by the tester's account. The two runs end differently, though: 2 of 12 jobs finished against 12 of 12, so the runtimes are not directly comparable. That nothing else changed is not established; the video description mentions two RTX PRO 6000 cards pulled.
Is more RAM worth it before a second GPU?+
When several people or agents use one model at the same time and RAM runs to its limit: probably yes, because RAM costs noticeably less per gigabyte than VRAM (we did not recompute prices). When a single user waits on speed alone, a faster card or a smaller model buys more.
senn-tech