AI Hardware

AI Hardware2026-09-22· By Franz Senn

M5 Ultra: the first measured numbers for local language models

Our comparison of Mac Studio, Strix Halo and DGX Spark on 6 September ended with a sentence that has followed us since: nobody had measured the M5 Ultra yet. Since 21 September there are measurements. Federico Viticci spent four days for MacStories putting an M5 Ultra with 256 GB against an M3 Ultra with 512 GB, Alex Ziskind lined both machines up with the same builds of llama.cpp and MLX, and Alex Grankin has compressed the published numbers of every review into one generation ladder. The spec sheet is not lying this time, and the number everyone is quoting only applies to half of all requests.

The Mac Studio with M5 Ultra ships from 22 September 2026, the first measurements for local language models landed on 21 and 22 September 2026. (Quelle: Apple / Wikimedia, public domain)

Alex Ziskind pits M5 Ultra against M3 Ultra: same llama.cpp and MLX builds, same model files, same prompts. Source of the GPU microbenchmark and the prompt length curve in this post.

The test setup

Ziskind runs the version with 80 GPU cores, 256 GB of memory and an 8 TB SSD for 14,299 US dollars, with the base model starting at 5,499. In Europe we sit at 12,429 euros, the price table is in our post from early September. The 512 GB tier follows at the end of October, the reference is an M3 Ultra with a stated 819 GB/s.

Mac Studio next to a Studio Display on a table — The Mac Studio chassis has not changed since 2022, including the airflow. The photo shows the previous generation next to a Studio Display. (Quelle: Wikimedia Commons, CC0)

Of the advertised 1,200 GB/s, 1,039 arrive

The instrument first, because the whole verdict hangs on it. STREAM runs over the CPU and hands the M5 Ultra 583.6 GB/s against 312.9 on the M3 Ultra, a real number and the wrong question, since inference runs on the GPU. The microbenchmark from the GPU side gives 1,039 GB/s on the M5 Ultra against 724 on the M3 Ultra. Both reach 87 to 89 percent of the factory figure, the shortfall is the same across the generation, and the ratio between the measured values is 1.44.

Both generations lose eleven to thirteen percent against the factory figure. What counts is the ratio between the measured values: 1.44. (Quelle: GPU microbenchmark from Ziskind's test run, Apple tech specs)

Three models, two physics

Generating text pulls every active weight through memory once per token, so throughput follows bandwidth. Reading the prompt is GPU compute, and that is where the M5 Ultra carries the Neural Accelerators. Those two curves can now be told apart.

Model	Type	Tokens/s M3 Ultra	Tokens/s M5 Ultra	Prefill M3	Prefill M5
DeepSeek V4 Flash, 284B	MoE	37	53	483	1,485
gpt-oss-120B	MoE	90	130	1,300	3,200
Qwen3.8-27B	dense	37	54	430	1,800

The generation column sits at a factor of 1.4 to 1.5 on all three models, the prefill column runs from 3.1 to 4.2 times. One side value says more about daily use than any peak: after 64,000 tokens already in context, another 32,000 cost 80 seconds on the M3 Ultra and 56 on the M5 Ultra.

Across the generations: three years of standing still, then a leap

Every review measures the same reference, Qwen3.8-27B at 4 bit. On that ladder M1, M2 and M3 Ultra all sit at around 30 tokens/s, the Ultra tier barely moved for three generations. The M5 Ultra sits at 46 to 54, 48 on average, the M5 Max at 30, an M4 Max at 23 and the new M6 mini at 10.

Bandwidth in brackets is the factory figure. M1 and M2 Ultra are listed at around 30 tokens/s, which are estimates fitted to community runs rather than measurements. (Quelle: Alex Grankin, aggregation of the reviews from 21 and 22 September)

Grankin converts the ladder into one job that cost his M5 Max 9.2 minutes and 16,500 tokens of output: 5.0 minutes on the 5090, 5.7 on the M5 Ultra, around 9 on M3 Ultra and M5 Max, 28 on the mini. Which makes the mini a bad entry into local models.

Alex Grankin collects the published measurements of the review sites into one generation ladder and scales them to a real job. He labels the M1 and M2 Ultra figures as estimates himself.

What long context does to speed

The four-fold better prompt processing is half the story: the same machine writes more slowly the more context it carries. Viticci and Grankin measure the same curve on Qwen3.8-27B: 48 tokens/s on a short prompt, 39 at 64,000, 32 at 128,000 and 24 at 256,000.

From 8,000 to 256,000 tokens of context the writing speed halves. 128,000 is the setting agent loops work with, and that is where 32 tokens/s sit. (Quelle: Grankin after MacStories and other reviews)

The second effect sits in memory: every context costs additional RAM, and that RAM is already committed. GLM 5.3 Flash writes 32 tokens/s on 256 GB but only fits up to 128,000 tokens of context, above that the machine runs full. Qwen3.8-Flash-Next writes 88 to 100 tokens/s and tolerates any context. The architecture decides, not the hardware alone.

Qwen3.8-27B at 4 bit is the reference of all four test runs: MacStories, Ziskind, Grankin and our own box work with the same file, which is why the numbers are comparable. (Quelle: Qwen / Hugging Face)

On the M5 Ultra. Those 100 seconds at 128,000 tokens of context were 250 on the M3 Ultra, and reading a whole code base still costs five minutes here. (Quelle: Grankin, MacStories)

The four only starts at four thousand tokens

Apple promises up to 4.5 times the peak AI compute of the M3 Ultra. Ziskind measured at which prompt length that becomes an actual four: 1.6 times at around 350 tokens, 3.4 times at 1,700, the four in the log from roughly 4,500, and on a real prompt of 14,000 tokens he measures 33 seconds against 8. Viticci measures 150 percent more prefill throughput on average and 245 versus 102 seconds to the first token behind a 256K prompt. Grankin's figure for an 80 page paste of 32,000 tokens: 19 seconds on the M5 Ultra, 39 on the M5 Max, 77 on the M3 Ultra.

llama.cpp now addresses the Neural Accelerators

The video shows the llama.cpp startup log, once per machine. On the M3 Ultra the runtime reports has_tensor = false, on the M5 Ultra has_tensor = true. That is the path to the matrix units, the reason for the prefill jumps. Our note from September about LM Studio concerned the runtime bundled there, and the bug report is still open. Anyone buying checks the build in the directory and its startup log.

The leap costs electricity

During generation the M5 tier reports around 45 watts for GPU plus CPU against 36 on the M3 Ultra, measured at the package and not at the wall. At idle it stays below the previous model at 8 to 10 watts. Under sustained full utilization the curve goes above 400 watts while the M3 Ultra stays near 200, so efficiency per token rises by about twelve percent. Add audible fans and 43 to 44 degrees of case temperature.

Mac Studio seen from above, air opening at the edge — Those 400 watts have to leave the box continuously, and the chassis has not changed since 2022. Photo: Mac Studio, previous generation. (Quelle: Wikimedia Commons, CC BY 2.0, photo Paul Hudson)

How our 4× RTX 5090 holds up against that

Viticci asked the same Qwen3.8-27B on all three worlds, as GGUF Q4_K_M on the card and as an MLX build on the Macs. On a 6,000-token prompt the 5090 reads 3,031 tokens/s and writes 59, the M5 Ultra 1,701 and 48, the M3 Ultra 414 and 31. With 64,000 tokens of context the three write 49.6, 38.9 and 23.5. The 5090 only keeps those numbers with an 8-bit attention cache inside its 32 GB; when it has to spill layers over PCIe, the same runs fall to 4.6 to 1.5 tokens/s. Grankin sees the same ranking: the card reads about twice as fast as the Mac, the Mac writes about three times as fast as a DGX Spark, which no longer works at 8 to 9 tokens/s with 128,000 of context.

Our box is not comparable to that, and the point belongs in the text: four RTX 5090 with tensor parallelism are 128 GB of VRAM and not a single stream. After four days of continuous operation with Qwen3.8-27B under vLLM we sit at an average of 96 tokens/s across all concurrent requests, with peaks around 16,000 tokens/s of prompt throughput. That is throughput for a group, not the response time of one desk. What the Mac brings is memory: 256 GB, no spilling, no PCIe edge, nearly silent.

Four RTX 5090 are 128 GB of VRAM spread over four cards and a PCIe edge. The Mac tier wins on contiguous memory, not on speed per card. (Quelle: NVIDIA)

Our take

The measurements confirm our reading in one line and Apple's claim in another: the generation gain is the bandwidth gain, and anyone who read the spec sheet was not surprised here. The actual leap sits in prefill, and it hangs on a condition that appears in no headline. Anyone who only chats locally buys an expensive M3 Ultra. Anyone who wants to run agents with long context on a workstation now has a machine worth calculating with. Two things stay open: the 512 GB tier at the end of October, and measurements beyond a single person, since eight concurrent requests are said to give 134 against 75 tokens/s while Viticci's three concurrent requests deliver a combined 81.5 against 39.9. So we are not buying, and we keep measuring.