AMD Strix Halo for Local LLMs

Strix Halo Is a Large-Memory PC, Not a High-Bandwidth GPU

AMD Strix Halo is the codename for the Ryzen AI Max 300 family. The flagship Ryzen AI Max+ 395 combines 16 Zen 5 CPU cores, a 40-CU Radeon 8060S integrated GPU, and up to 128 GB of LPDDR5X unified memory on a 256-bit bus.

That combination is unusually useful for local AI. A single compact x86 system can load models that exceed the VRAM of every consumer graphics card except the most expensive multi-GPU setups. It can also act as a normal Linux or Windows workstation without the Arm compatibility issues of NVIDIA DGX Spark systems.

Its limit is 256 GB/s of theoretical memory bandwidth. Strix Halo can hold very large models, but it cannot stream the weights of a large dense model quickly enough for fast single-user decode.

The best fit is therefore:

large quantized MoE models;
20B to 32B dense models;
long context and large KV caches;
a compact x86 development workstation;
local image, audio, and video experimentation through ROCm or Vulkan.

Key Specifications

Spec	Ryzen AI Max+ 395
Codename	Strix Halo
CPU	16 Zen 5 cores, 32 threads
CPU boost	Up to 5.1 GHz
GPU	Radeon 8060S, RDNA 3.5
GPU compute units	40
GPU clock	Up to 2.9 GHz
Unified memory	Up to 128 GB LPDDR5X
Memory interface	256-bit
Memory speed	LPDDR5X-8000
Memory bandwidth	256 GB/s theoretical
Configurable TDP	45–120 W
NPU	XDNA 2, up to 50 TOPS
Native connectivity	PCIe 4.0, two USB4 ports; networking depends on the system
Architecture	x86-64

AMD’s own Halo developer system uses 128 GB of memory, a 120 W power target, a 2 TB SSD, and 10 GbE. Retail Strix Halo machines vary widely: some use 64 GB, some expose only 2.5 or 5 GbE, and cooling quality changes sustained performance. For local LLMs, the 128 GB configuration is the reason to buy the platform.

Official specifications: Ryzen AI Max+ 395 and AMD Ryzen AI Halo developer platform.

The 256 GB/s Bandwidth Ceiling

A detailed Framework Desktop test measured about 215 GB/s of usable GPU memory bandwidth, roughly 84% of the 256 GB/s peak. That is a normal efficiency level for a shared LPDDR5X subsystem and shows that the published bandwidth is broadly attainable.

The issue is the amount of bandwidth.

Autoregressive decode at low batch sizes is usually constrained by how quickly the system can read model weights. A dense 70B Q4 model occupies roughly 40–45 GB, so each generated token requires moving a large weight set through a memory system that sustains a little over 200 GB/s under favourable conditions. Real results settle around 4–5 tokens per second.

The same machine can run a 100B-plus MoE model much faster because only a fraction of its experts are active for each token. Total parameters determine memory capacity; active parameters have a larger effect on decode speed.

Real Local-LLM Performance

The following results are representative rather than directly comparable. They use different llama.cpp versions, backends, quantizations, kernels, and firmware. They still show the platform’s operating range.

Model	Type	Quantization	Approx. decode
Qwen3 Coder 30B-A3B	MoE, 3B active	Q4_K_M	69–72 tok/s
GPT-OSS 120B	MoE	MXFP4	33–51 tok/s
Llama 4 Scout 109B	MoE, 17B active	Q4	19 tok/s
Qwen 3.5 122B-A10B	MoE, 10B active	MXFP4	18–20 tok/s
Qwen 2.5 Coder 32B	Dense	Q4_K_M	11 tok/s
Qwen3 32B	Dense	Q8_0	6.4 tok/s
Llama 3.x 70B	Dense	Q4 to Q6	3.8–5 tok/s
Qwen3 235B Thinking	MoE, 22B active	Q3_K_M, partial offload	7.8 tok/s

The newer high results use a tuned ROCm setup and a Strix Halo-focused llama.cpp fork. Earlier Framework testing produced similar dense-model speeds and slightly lower numbers on some MoE models. Software maturity can materially change prompt processing and kernel efficiency, but it does not remove the memory-bandwidth limit on dense decode.

Benchmark sources: Framework Strix Halo LLM tests and Strix-Halo-Models benchmark repository.

What Runs Well

MoE Models

Strix Halo is best when the model has a large total parameter count and a small active set. Qwen3 Coder 30B-A3B, GPT-OSS 120B, Llama 4 Scout, GLM Flash models, and Qwen 3.5 122B-A10B all make better use of the memory capacity than a dense 70B model.

Dense Models up to About 32B

Dense 14B to 32B models remain practical. A good 14B model can exceed 20 tok/s, while a dense 24B to 32B model generally lands around 10–15 tok/s at Q4. That is mildly usable for chat, coding, document analysis, and private agents.

Dense 70B models fit easily but are slow enough that many users will prefer a smaller model or an MoE alternative.

Long Context

Unified memory also gives Strix Halo room for KV cache, multimodal projectors, embeddings, and other runtime allocations. A model that nearly fills a 24 GB graphics card at short context has little margin for a 32K or 64K context window. A 128 GB system provides substantially more flexibility.

Context still consumes memory and can reduce prompt speed.

Software Support Is Better, but Still Uneven

For straightforward GGUF inference, Vulkan is often the least troublesome path. llama.cpp, LM Studio, and Ollama can use it without depending on every part of the ROCm stack being perfectly tuned for gfx1151.

ROCm is now an official option. AMD’s ROCm 7.2.1 compatibility matrix lists the Ryzen AI Max+ 395 and gfx1151 on Linux, with PyTorch 2.9.1 production support. AMD also publishes Windows support, although the Windows stack has more restrictions: only PyTorch is currently exposed, training is unsupported, distributed functions are limited, and AMD officially supports only LLM batch size one.

Linux is the stronger environment for vLLM, PyTorch, ComfyUI, and custom kernels. It may still require a recent kernel, correct shared-memory configuration, and model-specific tuning. AMD’s own documentation acknowledges lower-than-expected performance on some LLM workloads.

Useful references: ROCm Ryzen Linux support matrix, Ryzen limitations, and RDNA 3.5 system optimisation.

Clustering Is Usually the Wrong Upgrade

Strix Halo systems do not have a standard high-speed accelerator interconnect. Most machines expose 2.5, 5, or 10 GbE, while USB4 networking support and performance vary by operating system and driver.

A four-node Framework cluster demonstrated that large models can run, but current open-source distributed inference is inefficient. Llama 3.1 405B Q4 produced about 0.7 tok/s in one test. DeepSeek R1 671B Q2 reached 8.25 tok/s, while GPT-OSS 120B fell from 33 tok/s on one node to 24 tok/s across the cluster because the model already fit locally.

The forthcoming 192 GB platform is more useful for most local-LLM buyers than two 128 GB systems connected over ordinary Ethernet.

Cluster results: Jeff Geerling’s four-node Strix Halo test.

Gorgon Halo and the 192 GB Upgrade

AMD has now officially announced Ryzen AI Max PRO 400 processors, codenamed Gorgon Halo, for systems expected in the third quarter of 2026. The flagship Ryzen AI Max+ PRO 495 supports 192 GB of unified memory and up to 160 GB assigned as graphics memory.

Spec	Strix Halo 395	Gorgon Halo 495
CPU cores	16 Zen 5	16 Zen 5
CPU boost	5.1 GHz	5.2 GHz
GPU	Radeon 8060S	Radeon 8065S
GPU compute units	40	40
GPU clock	2.9 GHz	3.0 GHz
Maximum memory	128 GB	192 GB
Memory speed	LPDDR5X-8000	LPDDR5X-8533
Theoretical bandwidth	256 GB/s	~273 GB/s
NPU	50 TOPS	55 TOPS

Gorgon Halo is a capacity refresh. The same 256-bit bus at LPDDR5X-8533 raises theoretical bandwidth by only about 6.7%. Memory capacity increases by 50%.

That distinction matters for local LLMs:

models that already fit in 128 GB should decode only slightly faster;
larger MoE models, higher-quality quantizations, and longer contexts gain much more room;
dense models above 70B will fit at higher precision but remain bandwidth-bound;
192 GB reduces the need for slow multi-node inference.

Waiting for Gorgon Halo makes sense when the target workload needs more than roughly 90–115 GB of usable model and context memory. A discounted 128 GB Strix Halo system will remain the better value for 30B dense and 100B-class MoE models.

Official details: Ryzen AI Max+ PRO 495 specifications and AMD’s Ryzen AI Max PRO 400 announcement.

Strix Halo Versus DGX Spark

Strix Halo and NVIDIA GB10 have similar memory bandwidth, so their large dense-model decode speeds can be surprisingly close. Their strengths differ.

Strix Halo is usually cheaper, uses x86, runs Windows or Linux, and gives the CPU and GPU direct access to up to 128 GB. It is easier to treat as a normal workstation.

DGX Spark has CUDA, much stronger prompt processing, better support for NVIDIA-first development workflows, and 200 Gb/s ConnectX-7 networking. It is the better small cluster platform and the safer choice for software built around CUDA, TensorRT, or NCCL.

For a single local user running GGUF MoE models, Strix Halo offers more value. For CUDA development, high prompt throughput, image and video tooling, or multi-node experiments, GB10 has the stronger platform.

Buying Recommendation

Buy 128 GB Strix Halo for local LLMs when you want a compact x86 machine that can run large MoE models, dense models up to around 32B at useful speeds, and 70B models when capacity matters more than responsiveness.

Do not buy it expecting 128 GB to behave like 128 GB of HBM or GDDR6X. Dense decode follows bandwidth, and 256 GB/s is closer to DGX Spark than to an RTX 3090, RTX 5090, or a Mac Studio.

Wait for 192 GB Gorgon Halo when your models, quantization level, or context regularly exceed the practical memory allocation of the 128 GB systems. The extra memory is substantial.