Apple Silicon Macs for Local LLMs

Why Macs Work Well for Local LLMs

Apple Silicon combines the CPU and GPU with a large pool of unified memory. The GPU can access most of that memory directly, avoiding the 24–32 GB VRAM limit of consumer graphics cards.

The strongest Mac for local inference remains the M3 Ultra Mac Studio. Its 819 GB/s memory bandwidth gives it fast autoregressive decode, and discontinued configurations reached 512 GB of unified memory. The current M5 Max MacBook Pro has less capacity but adds much stronger prompt processing and 614 GB/s of bandwidth. The M4 Max Mac Studio sits between them, although Apple has removed its most useful 128 GB configuration from the store.

System	Maximum GPU	Memory bandwidth	Original maximum memory	Current Apple maximum
M4 Max Mac Studio	40 cores	546 GB/s	128 GB	64 GB
M5 Max MacBook Pro	40 cores	614 GB/s	128 GB	128 GB
M3 Ultra Mac Studio	80 cores	819 GB/s	512 GB	96 GB

Apple’s current specifications list only 64 GB for the M4 Max Studio and 96 GB for the M3 Ultra Studio. The older high-memory machines still exist, but they now have to be found through resellers or the used market.

Official specifications: Mac Studio and M5 Pro and M5 Max MacBook Pro.

Decode Is the Traditional Mac Strength

LLM inference has two main phases:

Prefill processes the input prompt and builds the KV cache. It relies heavily on matrix compute.
Decode generates tokens sequentially. At low batch sizes, it is usually constrained by memory bandwidth.

This makes the M3 Ultra good at interactive generation. A controlled EXO test used Llama 3.1 8B in FP16 with an 8,192-token prompt and a 32-token response. The M3 Ultra needed 5.57 seconds for prefill but only 0.85 seconds for generation. A DGX Spark completed prefill in 1.47 seconds but took 2.87 seconds to generate the same output.

The M3 Ultra has ample bandwidth but relatively modest matrix throughput. It starts responding slowly on long prompts, then produces tokens quickly. This is particularly visible with dense models and large contexts.

Source: EXO’s disaggregated prefill and decode benchmark.

M5 Improves Prompt Processing

M5 changes the balance by adding Neural Accelerators inside the GPU cores. These are separate from Apple’s conventional Neural Engine and can accelerate the matrix operations used by MLX.

Apple tested a base M5 MacBook Pro against M4 with 4,096-token prompts. Depending on the model, time to first token improved by 3.33–4.06×, while generation improved by 19–27%. The smaller decode gain tracks the increase from 120 to 153 GB/s on those base chips; the larger prefill gain comes from the new GPU compute path.

The high-end M5 Max scales this architecture to 614 GB/s and up to 128 GB of memory. In a separate M5-family test, Ollama’s MLX preview measured Qwen3.5-35B-A3B at 1,810 tok/s prefill and 112 tok/s decode with NVFP4, compared with 1,154 and 58 tok/s in its previous implementation. An int4 build reached 1,851 tok/s prefill and 134 tok/s decode. Ollama did not identify the exact M5 variant used for that chart.

Those results make the M5 Max MacBook Pro a more balanced local-LLM machine than earlier Max chips. It is also Apple’s only current Mac that can still be ordered with 128 GB.

Sources: Apple MLX research on M5 and Ollama’s MLX preview.

Representative LLM Results

These benchmarks use different models, quantizations, contexts, and runtimes, so they should not be treated as a direct ranking.

Hardware and model	Configuration	Result
M3 Ultra 512 GB, DeepSeek V3	671B, Q4, approximately 380 GB	20.2 tok/s decode
M3 Ultra, Llama 3.1 8B	FP16, 8K prompt	5.57 s prefill; 0.85 s for 32 output tokens
M5-family Mac, Qwen3.5-35B-A3B	NVFP4, Ollama MLX	1,810 tok/s prefill; 112 tok/s decode
Four M3 Ultra Studios, Qwen3 235B	EXO with Thunderbolt RDMA	Approximately 32 tok/s
Four M3 Ultra Studios, Kimi K2 Thinking	1T total, 32B active	Approximately 30 tok/s

A separate MLX distributed benchmark measured Kimi K2.5 at 16 tok/s across four Studios. Quantization, runtime, and parallelism strategy differ.

The M3 Ultra’s decode performance is close to what its memory bandwidth predicts. One large benchmark collection found single-node generation reaching roughly 95–111% of the simple bandwidth-over-model-size estimate. DeepSeek V3 Q4 is a good example: a model near 380 GB running at around 20 tok/s is already consuming most of an 819 GB/s memory system once overhead is included.

Benchmark references: MLX distributed benchmark collection and Jeff Geerling’s Mac Studio cluster.

MLX, Metal, and the Software Stack

MLX is the best native framework for Apple Silicon. It understands unified memory, supports efficient quantization, and includes distributed inference. mlx-lm is usually the first choice for testing recent Hugging Face models on a Mac.

Metal is Apple’s lower-level GPU API. llama.cpp, LM Studio, and traditional Ollama builds use Metal-backed kernels while retaining broad GGUF compatibility. This is convenient when a model is not available in MLX format.

Ollama is moving parts of its Apple backend to MLX, which improves prompt processing and enables newer low-bit formats. EXO builds on MLX and MLX Distributed to spread models across several Macs.

The trade-off: Macs do not support CUDA, and some image, video, training, and custom-kernel projects either arrive later or never receive a complete Metal implementation. Apple Silicon is strongest when the workload already runs well through MLX, llama.cpp, or a mature Metal application.

Pricing Changed Sharply in 2026

Apple increased US Mac prices on June 25, 2026 after first removing several high-memory configurations. The figures below are a July 11 snapshot; used listings vary with storage and GPU configuration.

System	Original US price	Current Apple price	Secondary-market snapshot
M4 Max Mac Studio, base	$1,999	$2,499	Discontinued 128 GB systems often listed around $7,000–$9,000
M3 Ultra Mac Studio, 96 GB base	$3,999	$5,299	Used or open-box asks roughly $5,000–$7,000
M3 Ultra, 80-core GPU, 512 GB, 1 TB	$9,499	Discontinued	Active asks commonly $20,000–$30,000; completed sales are sparse
14-inch M5 Max MacBook Pro	$3,599	$4,099	Market still thin
16-inch M5 Max line	$3,899 base	$4,399 base	40-core GPU/2 TB examples average about $4,152 sold on Swappa

The 512 GB M3 Ultra memory upgrade originally cost $4,000. Apple removed it in March 2026, then removed the 256 GB tier and the M4 Max Studio’s 128 GB option. Active marketplace listings now contain both extreme premiums and obvious scam patterns, so completed sales, seller history, serial verification, and buyer protection matter more than the lowest advertised price.

A fully configured 512 GB/16 TB M3 Ultra originally cost $14,099. Paying more than that can still make sense for a buyer who specifically needs a single 512 GB unified-memory node, but it is poor general workstation value.

Pricing references: 512 GB option removal, later memory reductions, June price increases, current M5 Max resale data, and M3 Ultra marketplace listings.

Clustering with EXO and Thunderbolt 5

macOS 26.2 added RDMA over Thunderbolt 5. EXO uses it with MLX Distributed to reduce communication latency and shard a model across multiple Macs.

Jeff Geerling tested four M3 Ultra Studios with 1.5 TB of combined memory. The cluster reached about 32 tok/s on Qwen3 235B and 30 tok/s on the one-trillion-parameter Kimi K2 Thinking model. EXO’s own repository shows four 512 GB Studios loading DeepSeek V3.1 at 8-bit and Kimi K2 Thinking at 4-bit.

The result is impressive, but it is not a simple replacement for one larger accelerator:

real Thunderbolt throughput was around 50–60 Gb/s;
there is no common Thunderbolt 5 switch, so four nodes require a dense cable topology;
RDMA must be enabled from recovery mode;
early testing included crashes, reboots, and failed model loads;
models that already fit on one Mac can lose performance from synchronization overhead.

EXO reports tensor-parallel scaling of up to 1.8× on two devices and 3.2× on four. The main reason to cluster is capacity: it allows a model too large for one machine to remain entirely local.

Sources: EXO and the four-node Thunderbolt 5 test.

Estimating a Possible M5 Ultra

Apple has not announced an M5 Ultra Mac Studio.

Ultra chips join two Max dies through UltraFusion and expose both memory controllers. The M3 Ultra therefore reaches 819 GB/s, approximately twice the bandwidth of a high-end M3 Max.

Applying the same relationship to the 614 GB/s M5 Max gives a rough estimate of 1.23 TB/s for an M5 Ultra. That would combine the M5 GPU’s much faster MLX prefill path with enough bandwidth to materially improve dense-model decode.

This estimate assumes Apple can fuse two full M5 Max dies and preserve their memory interfaces. It says nothing reliable about release timing, memory capacity, or price. Apple’s decision to skip M4 Ultra, remove high-memory configurations, and raise current prices makes all three uncertain.

Which Mac Makes Sense?

The M3 Ultra with 256 or 512 GB is still the strongest single Mac for very large local models. It can run DeepSeek-class 671B quantizations without a cluster and offers excellent decode, but resale pricing has become difficult to justify.

The M5 Max MacBook Pro with 128 GB is the best current Apple configuration for a balanced mix of prompt processing, decode, portability, and supported memory. It suits 70B dense models, large MoEs, coding models, and local development, but sustained workloads pay for laptop packaging and thermals.

The M4 Max Mac Studio remains a quiet, efficient desktop with up to 546 GB/s. The current 64 GB limit makes it less compelling for large-model inference. A reasonably priced used 128 GB system is far more interesting than the model Apple sells now.

For a cluster, use Thunderbolt 5 and EXO only when the model cannot fit on one node. For most buyers, one high-memory Mac is simpler, more stable, and usually faster.

For a x86 alternative with similar unified-memory capacity, see AMD Strix Halo for Local LLMs.