Topics

Local AI and GPU hardware

Name: senn-tech
Address: Weissachstraße 67, Kufstein, Tirol, 6330, AT
Telephone: +4369917602100
Price range: €€

Posts on running language models in-house: GPUs, mini workstations and inference servers, with measurements for throughput, power draw and cost per token.

33 posts

2026-10-24 · HardwareOlares One first look: AI mini PC with 24 GB VRAM, not 32The Olares One puts an RTX 5090 Mobile with 24 GB into a 3.5-litre case. Caschy published a first look on 24 October. We work out which models fit in memory and separate vendor figures from what can be checked.
2026-10-24 · HardwareFSR 4.1 on Radeon RX 6000: tested on RDNA 2 with INT8Driver 26.10.1 brings FSR Upscaling 4.1 to the RDNA 2 generation. ComputerBase sees clearly better image quality, but 10 to 19 percent fewer FPS than FSR 3.1 on the RX 6950 XT. Technique, measurements and open questions.
2026-10-19 · AI HardwareRTX 5090 not discontinued, yet €6,800 to €9,999: what it means for local LLM buildsNvidia denies the RTX 5090 is ending, German retail is nearly out of stock, and the last cards cost three times the list price. A calculation for anyone planning a local LLM machine.
2026-10-19 · AI HardwareLlmspeed measured: the speed estimate for local LLMs and our numbers on the R9700 and RTX 5090An open-source tool promises to make the speed of local models tangible before a purchase. We put its estimates for the R9700 and RTX 5090 against our two-card run from October 16.
2026-10-18 · AI HardwareQwen3.8-Flash-Next on vLLM 0.32: MTP doubles single-user speed, the cache gets smallervLLM 0.32 runs Flash-Next natively. On our four RTX 5090 a single user's reply speed goes from 104 to 217 tokens per second, while the cache for concurrent sessions shrinks from 8.2 to 6.1 full-length contexts. This is what the move took and what nearly went wrong.
2026-10-18 · AI HardwareAI data centres stall, neoclouds under pressure: what it means for rented GPU capacity47 large US projects delayed, cancelled or blocked, CoreWeave down about 25 percent in a week, and Gartner expects 80 percent of neoclouds to lose their independence. What is documented, and what it means if you rent GPU capacity.
2026-10-18 · AI InfrastructureBifrost vs LiteLLM: an open-source AI gateway and its 50x speed claimBifrost offers an OpenAI-compatible API, failover, budgets and MCP, and is said to be 50 times faster than LiteLLM. The figure comes from the project website; the GitHub page tells a different story. How to test it yourself.
2026-10-17 · AI HardwareRTX Spark After Launch: First Measurements, the Variant Trap, and the Linux QuestionSince October 16, 2026 the RTX Spark laptops are in stores, from 2,599 US dollars per the press release. Two labs measure 14.5 to 89 tokens per second, NVIDIA's own pages contradict each other, and the memory decides whether a workload runs at all.
2026-10-16 · AI HardwareRadeon R9700 against RTX 5090: same file, two cards, one price per token120 to 146 tokens per second on the RTX 5090 against 41 to 45 on the Radeon R9700, same model file, same engine. The prices sit 3.10 apart. What is left over is watts and capacity.
2026-10-16 · AI HardwareMeta Muse Spark Open: an 816B model for local use, and what the hardware really takesMeta released Muse Spark Open: 816 billion parameters, 42 billion active, 1 million tokens of context. Locally it runs, per Meta, on two Mac Studios with 512 GB each, which are expected only at the end of October.
2026-10-16 · HardwareLedger wallets with a spy board: what is documented about the hardware implant and how to buy devices safelyLedger wallets carried an extra board with an LTE modem and eSIM that reads the recovery phrase. Loss estimates range from 86 to 102 million dollars, and the route into retail is open. What the sources document and what applies to any security device.
2026-10-15 · AI HardwareNvidia RTX Spark (N1X) tested: how fast do local LLMs run on the notebook chip?Nvidia's first notebook chip offers up to 128 GB of shared memory. c't measures 14.5 tokens/s for a dense 27B model and almost 61 for a MoE. Prices from 2,899 euros make comparisons with Strix Halo and Mac awkward.
2026-10-13 · ProsumerApple HomeView, HomePod mini 2 and Apple TV 4K: prices, dates, not for the EUOn 13 October 2026 Apple showed HomeView, a smart home hub with a display, plus a HomePod mini 2 and an Apple TV 4K. Prices, dates, hardware and the gaps: no EU sales, and the sources are silent on Matter and Thread.
2026-10-11 · AI HardwareToken prices fall, GPU rental stays high: Jevons in the AI marketThe cheaper a token gets, the more of them are used, and GPU demand stays high. What a16z shows, which numbers are actually reported, and what follows for renting versus buying.
2026-10-05 · AI HardwareTwo GB300 towers, two 400G copper cables: what a second box actually multipliesStorageReview cabled an ASUS and an HP GB300 tower together with two 400G DACs and measured the pair. The data shows where a second box scales by an order of magnitude and where only two- to three-fold gains remain.
2026-10-03 · HardwareMemory prices in Q4 2026: enterprise SSDs up as much as 28 percent, Micron expects 2027 to be tighterTrendForce expects Q4 2026 contract prices for DRAM to rise 10 to 15 percent over the previous quarter, NAND 15 to 20 percent and enterprise SSDs 23 to 28 percent. On the same day Micron reported USD 54.23 billion in quarterly revenue and said supply will be tighter in 2027 and 2028 than in 2026. For server and SSD purchases for projects starting in 2027, that raises the question of when to order.
2026-10-03 · KI-HardwareNVIDIA fixes 114 flaws in its GPU driver and vGPU: which version secures which branchOn 30 September 2026 NVIDIA published a security bulletin with 114 CVEs for its GPU drivers and vGPU software, 78 rated High and 36 Medium. On Linux the fixed releases are 615.71.09, 610.57.04, 595.91.07 and 580.178.04, for vGPU they are 20.2 and 19.6. Almost every flaw needs local access, and the fix only takes effect once the kernel module is reloaded.
2026-10-03 · HardwareLaptops under 800 euros: HP OmniBook 5 with Wildcat Lake and Lenovo V15 with Snapdragon C against the MacBook NeoOn 1 October 2026 HP announced the OmniBook 5 14 with Intel Wildcat Lake from 699.99 US dollars, and on 30 September Lenovo confirmed the V15 G7 Q7X with Snapdragon C, still without a price. The MacBook Neo costs 799 euros in Austria and 699 dollars in the US, the same as the HP. Notebookcheck measured the Wildcat Lake chip at the level of a Core i7-1355U, and HP has published neither a euro price nor a date for Germany, Austria or Switzerland.
2026-10-03 · AI HardwareTwo DGX Spark boxes on our bench: DeepSeek V4-Flash runs, V4.1-Flash does not fitTwo linked DGX Spark boxes give you 256 GB of shared memory and 546 GB/s of bandwidth. DeepSeek V4-Flash runs on that. The newer V4.1-Flash does not fit, and the reason is a memory trick that saves nothing on this kind of hardware.
2026-10-02 · KI-HardwareKV cache in 4 bits: INT4 failed in production, NVFP4 holds 8.2 full sessionsOn 2 October we switched the KV cache of Qwen3.8-Flash-Next to INT4, on 3 October we switched back: in production 3 of 248 model steps were garbage, although every check had passed. Since 3 October the lane runs NVFP4 with a pool of 1,810,884 tokens, 8.2 full sessions of 220,000 tokens against 5.2 with FP8. This post covers both days and the new check.
2026-09-29 · KI-HardwareCUDA 13 becomes the floor: vLLM 0.30.0 and llama.cpp 0.5.0 moved their build basevLLM 0.30.0 of 22 September builds against CUDA 13.0 by default, llama.cpp v0.5.0 of 23 September ships CUDA 13.4 builds and moves the RPC protocol to major v7. For any operator that means: measure the driver before pulling the image, and keep llama.cpp in lockstep across hosts. On our fallback lane 0.30.0 runs on driver 610.57.04, measured 29 September.
2026-09-24 · KI-HardwareTwo vLLM builds for Qwen3.8-Flash-Next: faster decode, tighter memoryThe vLLM nightly from 24 September keeps the Qwen3.8-Flash-Next KV cache in FP8 without a local patch. On 4× RTX 5090 it decodes almost 50 percent faster than our patched production build, but it runs into the memory limit under long parallel requests. For now, production stays on the old build.
2026-09-22 · AI HardwareM5 Ultra: the first measured numbers for local language modelsThe first M5 Ultra measurements landed on 21 and 22 September: 1,039 of the advertised 1,200 GB/s, decoding 1.4x faster, prefill 3 to 4x. Plus the condition Apple writes next to that number.
2026-09-21 · AI HardwareASUS Ascent QN10 reviewed: 18 cores, an 80 TOPS NPU and no software that reaches itASUS built an 18-core mini PC for local AI agents and shipped an 80 TOPS NPU with it. The measurements show that the big number never arrives where the models actually run.
2026-09-18 · KI-HardwareRTX 5090 sold out, RTX Pro 5500 with 84 GB: recalculating the purchaseThird-party sellers are asking up to USD 9,500 for the RTX 5090, while NVIDIA's RTX Pro 5500 offers 84 GB on the same chip. The new calculation is based on bandwidth and capacity, not price.
2026-09-14 · AI HardwareOne word in the vLLM fork: why our Flash-Next lane paid for every token three times overAn agent set out to squeeze more concurrent sessions out of our Qwen3.8-Flash-Next lane, knocked over production doing it, and got the cause wrong. What was right was its measurement: every token cost 2.6 times as much KV cache as the block geometry allows. The cause was a single word in the code of the vLLM fork.
2026-09-09 · AI HardwareQwen3.8-Flash-Next in production: how a community patch doubled the KV cache on 4× RTX 5090A week ago the verdict was: stable, but no production lane. Since this week, Qwen3.8-Flash-Next serves every agent lane we run. Getting there went through a hand-merged hybrid checkpoint, a bad measurement, a queue backlog, and a patch out of an open GitHub ticket.
2026-09-08 · AI HardwareMac Studio M5 Ultra, Strix Halo or DGX Spark: which box really makes local language models fastThree desktop boxes with 128 GB or more of unified memory are courting the same buyers. The spec sheets talk about TOPS and petaflops, the measurements talk about bandwidth: 1,200 versus 273 versus 256 GB/s. On top of that, a year in which every box got more expensive, and a successor with 192 GB on show at IFA.
2026-09-03 · AI HardwareQwen3.8-Flash-Next, four days in: why production stays on the 27BAfter the Qwen3.8-Flash-Next announcement, we actually got the model running on our 4×RTX 5090 host: a custom llama.cpp build, two crashes, a RAM upgrade, and a verdict that ended up resting on a different reason than we expected on day one.
2026-09-01 · AI Hardware4× RTX 5090: When DRAM Bandwidth Is the Real BottleneckFour RTX 5090s with 128 GB VRAM sounds like overkill. The surprise on our kipc5090 host: the limiting factor is not GPU compute, but the memory bus, because only two of six DDR5 channels are populated.
2026-08-03 · Hardware & GadgetsVivo X300 Ultra: Five days with the camera flagshipThe Vivo X300 Ultra after five days of use: The photos are outstanding, and the Android experience is a pleasant surprise. The camera module makes it noticeably top-heavy, but for a China phone, the overall package is genuinely impressive.
2026-07-29 · Prosumervivo X300 Ultra vs. iPhone, Galaxy, and Pixel: Why We're Buying This Camera FlagshipA camera flagship from China takes on the Western high-end market: What the vivo X300 Ultra does better for 1,999 euros, and what you give up in return.
2026-07-28 · Prosumer5K Displays for the Mac: Studio Display, XDR, or BenQ?Apple has re-released the Studio Display in 2026, but the panel remains unchanged. What this means for your purchasing decision, and when a BenQ is the better choice.

Local AI and GPU hardware

Other topics