Apple M5 Ultra for Local AI

A Shift in Apple Silicon Workstations

Apple’s M5 Ultra announcement revealed up to 80 GPU cores, a 32-core Neural Engine, and Neural Accelerators in every GPU core. The chip provides 1,228 GB/s of unified memory bandwidth and up to 512 GB of unified memory for not much more money than an rtx 6000 pro. For local AI, that combination is insane.

Apple Silicon has long offered key benefits for local large language models (LLMs): unified memory shared directly between CPU and GPU, modest power consumption, and quiet desktop operation. Earlier Ultra chips had an uneven performance profile, however. They generated output tokens quickly during decode thanks to ample memory bandwidth, but prompt processing (prefill) was slow because older Apple GPUs lacked dedicated matrix hardware. The M5 generation addresses this imbalance by integrating Neural Accelerators directly into each graphics core, paired with an UltraFusion interconnect that delivers that beautiful 1.2 TB/s of unified memory bandwidth.

Why I Preordered the M5 Ultra

I cancelled preorders for two NVIDIA DGX Spark systems to preorder the M5 Ultra Mac Studio instead.

I chose the configuration with the 80-core GPU and 256 GB of unified memory. My primary reason is that I get multiples more bandwidth for similiar money and no networking bottleneck.

Two DGX Sparks would have provided 256 GB of combined memory across two boxes linked over 200 Gb/s networking. In practice, clustering two physical nodes introduces operational friction: network latency across sharded layers, software setup on Arm-based Linux, and each node’s 273 GB/s memory bandwidth limit.

By contrast, the M5 Ultra puts 256 GB into a single desktop box with 1,228 GB/s (1.2 TB/s) of memory bandwidth—roughly four and a half times that of a single Spark. If the M5 architecture resolves the historical prompt-processing bottleneck, a single Mac Studio offers a simpler, quieter, and more practical daily workstation.

Availability and Delivery Dates

Apple’s official schedule sets general availability for September 22, 2026 for 96 GB and 256 GB configurations. The top 512 GB configuration is scheduled to ship later, in late October 2026.

I ordered the 80-core GPU configuration with 256 GB of unified memory. My preorder confirmation estimates delivery in October or November.

How UltraFusion Doubles Memory Bandwidth

The M5 Ultra connects two discrete M5 Max dies into one large processor using Apple’s UltraFusion packaging technology, a high-density silicon bridge providing massive cross-chip throughput with minimal latency.

Bandwidth doubles because of the physical memory controllers:

1. A single M5 Max die has a 512-bit memory bus delivering 614 GB/s of peak theoretical bandwidth.
2. Fusing two M5 Max dies runs both sets of memory controllers concurrently, doubling the physical bus to 1024 bits.
3. Operating both memory systems in parallel doubles theoretical bandwidth from 614 GB/s to approximately 1.2 TB/s (1,228 GB/s).

Unlike conventional PCs that transfer weights across a PCIe slot, Apple’s Unified Memory Architecture lets the GPU access the entire 256 GB pool directly without copy overhead.

Estimating Prompt-Processing Speed

LLM inference has two distinct phases:

• Prefill (Prompt Processing): The model processes the prompt in parallel to construct its key-value cache. This compute-bound stage depends on raw mathematical throughput.
• Decode (Token Generation): The model generates output text one token at a time, streaming weights from memory for every token. This stage is almost always memory-bandwidth-bound.

The Previous Bottleneck

On earlier Apple chips, prompt processing lagged behind dedicated accelerators. In a test published by EXO using Llama 3.1 8B in FP16 with an 8,192-token prompt, an M3 Ultra Mac Studio took 5.57 seconds to process the prompt (~1,470 prompt tokens per second). An NVIDIA DGX Spark took 1.47 seconds for the same prompt (~5,572 prompt tokens per second).

Despite having three times the memory bandwidth of the Spark, the M3 Ultra was nearly four times slower during prefill because its graphics cores handled matrix math with general-purpose units.

The M5 Neural Accelerators

The M5 family introduces dedicated Neural Accelerators inside every GPU core. In official product disclosures, Apple states that the M5 Ultra delivers up to 4x faster prompt-processing performance compared to the M3 Ultra in LLM applications like LM Studio.

Applying that four-fold improvement to the measured M3 Ultra baseline provides a straightforward estimate: multiplying 1,470 tokens per second by four suggests that an 80-core M5 Ultra could process an 8B model prompt at around 5,900 tokens per second.

Because runtimes, quantization formats, and prompt lengths vary across real-world setups, this figure is a rough estimate.

Decode Expectations vs. NVIDIA Blackwell Cards

During token generation, speed is dictated by memory bandwidth: how quickly the processor can stream model weights from memory.

Comparing official specifications gives clear spec-derived expectations:

• Apple M5 Ultra: 1,228 GB/s (1.2 TB/s unified memory)
• NVIDIA RTX PRO 5000 Blackwell: 1,344 GB/s (48 GB GDDR7)
• NVIDIA RTX PRO 6000 Blackwell: 1,792 GB/s (96 GB GDDR7)

Comparing these published numbers indicates:

• The M5 Ultra’s 1,228 GB/s is approximately 68.5% of the RTX PRO 6000 Blackwell, or about two-thirds the bandwidth.
• The M5 Ultra’s 1,228 GB/s is approximately 91.4% of the RTX PRO 5000 Blackwell, or just under its bandwidth.

These ratios are theoretical expectations based on bus width and clock rates, not measured head-to-head benchmarks. Real-world generation speeds depend on software drivers and runtime efficiency. Mature CUDA runtimes extract high efficiency on NVIDIA cards, while Apple’s frameworks must coordinate memory access across two fused dies.

Crucially, workstation cards face video memory ceilings: 48 GB on the RTX PRO 5000 and 96 GB on the RTX PRO 6000. While Blackwell cards generate tokens faster on models that fit inside their VRAM, an M5 Ultra with 256 GB or 512 GB can run much larger models that cannot load onto a single workstation GPU at all.

You can model fit across these systems using the Tokens-per-Second Calculator.

MLX and Software Maturity

Hardware specifications only translate into delivered speed if software takes advantage of them. On Apple Silicon, the core framework is MLX, developed by Apple’s machine learning research team.

MLX is tailored for unified memory, using zero-copy data structures to share data between CPU and GPU without staging buffers. It also applies lazy evaluation to combine mathematical operations into single GPU tasks, saving memory round-trips.

As MLX matures, future updates can extract additional performance from the M5 Ultra by scheduling computations to keep data local to each die, adding faster kernels for low-bit quantization, and managing memory efficiently during long-context sessions.

Hardware Comparison

System	GPU Cores	Memory Bandwidth	Max Memory	Main Strength	Key Limitation
Apple M4 Max Studio	40	546 GB/s	64 GB	Modest entry price	64 GB memory ceiling
Apple M5 Max MacBook Pro	40	614 GB/s	128 GB	Portable, fast prefill	Laptop cooling; 128 GB limit
Apple M3 Ultra Studio	80	819 GB/s	512 GB	Fast decode on large models	Slow prefill on long prompts
2x NVIDIA DGX Spark	12,288 CUDA	2x 273 GB/s	256 GB	Native CUDA, Blackwell FP4	Network complexity; 273 GB/s decode limit
Apple M5 Ultra Studio	80	1,228 GB/s	512 GB	Balanced speed, huge memory, one box	Starts at $5,499; no native CUDA

For interactive local AI, 256 GB of memory in a single machine avoids the operational friction of running models across a network cluster.

Summary

The M5 Ultra Mac Studio provides an appealing balance of hardware for local AI. By doubling memory bandwidth to 1.2 TB/s and introducing Neural Accelerators to resolve its long-standing prompt-processing bottleneck, Apple has built a capable single-node workstation.

For developers and researchers running larger dense models or 200B-plus Mixture-of-Experts architectures, the 256 GB and 512 GB memory configurations keep complete models resident in fast memory. While dedicated NVIDIA graphics cards remain the standard for high-volume cloud servers, the M5 Ultra offers an unmatched combination of capacity, bandwidth, and affordability for private local inference.

Further Reading

• Apple Introduces New Mac Studio with M5 Max and M5 Ultra
• Mac Studio Specifications
• Exploring LLMs with MLX and Neural Accelerators on M5
• EXO Disaggregated Prefill and Decode Benchmark
• Apple Silicon Macs for Local LLMs
• NVIDIA DGX Spark and GB10 for Local LLMs
• Best GPUs and Devices for Local AI
• Interactive Tokens-per-Second Calculator