NVIDIA DGX Spark and GB10 for Local LLMs

DGX Spark Is a Slow Capacity Machine

NVIDIA DGX Spark is one of the strangest and most interesting local AI systems available.

It is a tiny 150 mm desktop containing the GB10 Grace Blackwell Superchip, a 20-core Arm CPU, a Blackwell GPU, 128 GB of coherent LPDDR5X unified memory, CUDA support, and a ConnectX-7 200 Gb/s network interface. NVIDIA’s Founders Edition includes a 4 TB NVMe SSD, while partner systems from ASUS, Acer, Dell, HP, Gigabyte, Lenovo, and MSI offer different storage, cooling, support, and pricing.

The headline capacity is excellent. A single box can load models that are impossible on a 24 GB or 32 GB consumer GPU. Two to four systems can be connected over RoCE to run models that normally require a server.

The catch is equally important: the unified memory delivers only 273 GB/s of theoretical bandwidth. That is enough to fit huge models, but not enough to decode dense models quickly. A used RTX 3090 has 936 GB/s. Apple’s M3 Ultra has 819 GB/s. Even the current M5 Max MacBook Pro reaches 614 GB/s.

It is a low-power CUDA development and inference appliance for large quantized models, especially MoE models, batched jobs, and clustered experiments. It is not a replacement for a high-bandwidth discrete GPU when single-user dense-model decode speed is the priority.

Key Specifications

Spec	NVIDIA DGX Spark
SoC	NVIDIA GB10 Grace Blackwell Superchip
CPU	20-core Arm: 10 Cortex-X925 + 10 Cortex-A725
GPU	Blackwell, 6,144 CUDA cores, 5th-gen Tensor Cores
AI marketing figure	Up to 1 PFLOP FP4 with sparsity
Unified memory	128 GB LPDDR5X
Memory interface	256-bit, LPDDR5X-8533
Memory bandwidth	273 GB/s theoretical
NVIDIA storage	4 TB NVMe M.2, self-encrypting
GB10 platform storage	1 TB or 4 TB depending on system; some OEMs also offer 2 TB
Networking	10 GbE plus ConnectX-7 at 200 Gb/s
ConnectX ports	2x QSFP
Wireless	Wi-Fi 7, Bluetooth 5.4
Media engines	1x NVENC, 1x NVDEC
USB / display	4x USB-C, HDMI 2.1a
GB10 TDP	140 W for the SoC
Power supply	240 W external PSU
Dimensions	150 × 150 × 50.5 mm
Weight	1.2 kg
Operating system	DGX OS, based on Ubuntu 24.04 on Arm64

NVIDIA’s lists 128 GB on a 256-bit interface at 273 GB/s, 6,144 CUDA cores, a 140 W GB10 TDP, and a 240 W system power supply. The 1 PFLOP number is a theoretical FP4 result that assumes structured sparsity; it is not a dense BF16 performance number and should not be used to predict LLM token speed.

Official specifications: NVIDIA DGX Spark hardware guide and DGX Spark product page.

Is the 273 GB/s Bandwidth Fully Usable?

Mostly, under the right workload.

A community CUDA test using large 128 MB buffers measured about 229–230 GB/s for copy and triad operations on an idle Spark. That is roughly 84% of the 273 GB/s theoretical maximum, which is a credible real-world result for LPDDR memory after protocol, controller, coherence, and access-pattern overhead.

The memory subsystem can get reasonably close to its advertised peak.

The problem appears during real inference. In the same investigation, active vLLM decode reduced the residual bandwidth available to a concurrent test by around 55–60%. Profiling suggested that optimized GEMM and MoE kernels were already consuming most of the memory traffic. It was hitting the architectural bandwidth ceiling.

Benchmark details: NVIDIA developer forum bandwidth investigation.

Why Dense Decode Is Slow

LLM inference has two different phases.

Prefill processes the prompt in parallel. It is compute-heavy, especially with long prompts, and benefits from Blackwell Tensor Cores, FP4 support, CUDA kernels, and large matrix multiplication throughput.

Decode produces output one token at a time. At low batch sizes, the accelerator repeatedly reads a large portion of the model weights for each generated token. This is usually memory-bandwidth bound.

A rough upper-bound model:

decode tokens/s ≈ usable memory bandwidth / bytes of active weights per token

This is deliberately simplified. Attention, KV-cache traffic, quantization metadata, kernel efficiency, expert routing, cache hits, and speculative decoding all affect the result.

A dense 70B Q4 model may occupy around 40–45 GB after quantization and metadata. Streaming that weight set through a 273 GB/s memory system gives a low single-digit theoretical range once real overhead is included.

The GB10 GPU has far more compute than dense single-stream decode can feed. The bottleneck is the memory path.

Real Single-Node LLM Performance

Ollama published a useful controlled benchmark using firmware 580.95.05, Ollama 0.12.6, ten runs per model, a 500-token output, disabled caching, and a fixed prompt.

Model	Quantization	Prefill tok/s	Decode tok/s	Model type
gpt-oss 20B	MXFP4	3,224	58.27	MoE
gpt-oss 120B	MXFP4	1,169	41.14	MoE
Gemma 3 12B	Q4_K_M	1,894	24.25	Dense
Gemma 3 12B	Q8_0	1,406	15.46	Dense
Gemma 3 27B	Q4_K_M	834	10.83	Dense
Gemma 3 27B	Q8_0	585	7.21	Dense
Llama 3.1 8B	Q4_K_M	7,614	38.02	Dense
Llama 3.1 70B	Q4_K_M	1,911	4.42	Dense
DeepSeek-R1 14B	Q4_K_M	5,919	19.99	Dense
Qwen3 32B	Q4_K_M	705	9.41	Dense

Source: Ollama’s NVIDIA DGX Spark performance tests.

These numbers summarise the platform better than the petaflop marketing:

Prompt processing is often extremely fast.
Small dense models are fine.
Large dense models fit but decode slowly.
Large sparse MoE models can be dramatically faster because only a small subset of experts is active for each token.
Quantization format and backend matter as much as the total parameter count.

LMSYS measured Llama 3.1 70B FP8 at about 803 tok/s prefill and 2.7 tok/s decode, reinforcing the same conclusion: strong prompt processing, weak dense decode. See the LMSYS DGX Spark review.

MoE Is the Best Match for GB10

Mixture-of-Experts models contain a large total parameter count but activate only a subset of experts for each token.

A 120B MoE model with perhaps 5–15B active parameters can use the 128 GB capacity for model quality while moving far fewer active weights during each decode step than a dense 120B model. The result can be tens of tokens per second rather than low single digits.

The current vLLM guidance for Spark describes 100–130B NVFP4 MoE models with roughly 10–15B active parameters as a strong fit. It also warns that the operating system, CPU, GPU, container runtime, model weights, page cache, and KV cache all share the same 128 GB pool. You cannot safely allocate the full nominal capacity to vLLM.

Source: vLLM on DGX Spark: architecture and configuration.

Good single-Spark targets include:

gpt-oss-120B MXFP4
Nemotron 3 Super 120B-A12B NVFP4
Qwen 35B A3B MoE with a small active set
Other 100–200B-class NVFP4 or MXFP4 MoE models that have a tested GB10 kernel path

The phrase tested kernel path matters. A theoretically good model can perform badly if its quantization, attention implementation, expert kernel, or speculative decoder is not optimized for sm_121.

Concurrency Changes the Value Proposition

A single chat stream is close to the worst way to benchmark this machine.

During batched inference, the system can reuse a model-weight read across multiple sequences. Aggregate throughput can therefore rise much faster than per-sequence latency falls, until compute, KV cache, scheduling, or memory capacity becomes the new limit.

One independent vLLM benchmark reported the following results on a single GB10 system.

Nemotron Super 49B NVFP4

Concurrent sequences	Aggregate tok/s	Per-sequence tok/s
1	5.79	5.79
8	45.41	5.68
32	161.90	5.08
128	490.68	3.92
256	695.11	2.85

The same test reported gpt-oss-120B at about 33.5 tok/s for one sequence and 862.8 tok/s aggregate at concurrency 256.

Source and methodology: Dendro Logic DGX Spark concurrency benchmark.

Treat these as one workload on one software stack, not a guarantee. The benchmark used structured extraction prompts, moderate output lengths, and a specific NVIDIA vLLM container. Very long contexts, hybrid state-space models, multimodal inputs, and different kernels can scale differently.

It still proves an important point: the Spark can be a much better throughput server than single-user chat.

Useful high-concurrency workloads include:

Bulk document extraction and classification
Parallel code review or test-generation agents
Multi-agent workflows that fan out into independent tasks
Local API serving for a small team
Evaluation harnesses and synthetic data generation
Running a large reasoning model and a smaller utility model at the same time

For one person chatting with one dense model, buy bandwidth. For many independent requests against a large MoE model, Spark is justifiable.

Dual Spark: More Than 256 GB of Memory

Two DGX Spark systems can be connected directly with a QSFP cable over their ConnectX-7 interfaces. NVIDIA now provides a cluster assistant in NVIDIA Sync for two- to four-node systems.

Two nodes provide:

256 GB of aggregate unified memory
Two memory controllers, giving useful aggregate bandwidth when the model is partitioned well
200 Gb/s RoCE between nodes
Enough capacity for many 250–400B-class quantized models

The network number needs translating. 200 gigabits per second is 25 gigabytes per second theoretical, before protocol and software overhead. It is much slower than the 273 GB/s local memory path.

A cluster therefore does not become one 256 GB memory pool with local-memory latency. Good distributed inference keeps each weight shard local and sends activations, routing data, or collective-reduction traffic across the network. A poor tensor-parallel layout can spend too much time communicating and lose most of the benefit.

NVIDIA supports direct two-node links, a three-node ring, and two-to-four nodes through a suitable RoCE switch. The official guidance calls for 200–400G-capable ports and at least 0.8–1.6 Tbps of switching capacity for switched topologies.

Source: NVIDIA’s multi-node DGX Spark guide.

DeepSeek V4 Flash on Two Sparks

DeepSeek V4 Flash is one of the strongest demonstrations of the platform’s intended use.

A community-validated recipe uses:

2x DGX Spark
Direct 200G QSFP56 RoCE/NCCL connection
The official DeepSeek V4 Flash FP8 weights, roughly 149 GB across 46 shards
Tensor parallelism across two nodes
Expert parallelism
FP8 KV cache
Multi-Token Prediction
A pinned vLLM commit with GB10-specific validation

Warm single-stream decode was reported around 35–44 tok/s, depending on the test, with roughly 800–1,100 tok/s prompt processing at short-to-moderate contexts. At 200K context, the same recipe had much slower time to first token, and concurrency had to be reduced to preserve KV-cache capacity.

Source and reproducible discussion: DeepSeek V4 Flash on 2x DGX Spark.

Newer custom speculative-decoding integrations report roughly 60 tok/s at up to around 900K context on two Sparks. They use a specialised DSpark path and should not be compared directly with the stock FP8/MTP recipe.

Four Sparks: GLM-5.2 Is Possible, With Caveats

Four GB10 systems provide 512 GB of aggregate memory.

Community results for GLM-5.2 show several different trade-offs:

A fuller GLM-5.2 IQ4_XS quant across four GB10 nodes has been reported at about 6.28 tok/s decode with DeepSeek Sparse Attention active.
A heavily tuned four-node build using 15% expert pruning, AWQ INT4, custom sparse-MLA work, and MTP has reported around 20–22 tok/s decode at up to 256K context.
A three-node 469B REAP-pruned NVFP4 build reports around 4.4 tok/s decode and 2,500–3,800 tok/s prefill at 256K context.

Sources: full GLM-5.2 IQ4_XS on 4x GB10, optimised GLM-5.2 on a 4x GB10 cluster, and GLM-5.2 469B on 3x DGX Spark.

These are not equivalent benchmarks. The fastest result changes the model through pruning and uses custom optimisations.

The conclusion is still impressive: four small, low-power boxes can run a frontier-scale MoE model locally at usable speed and for a fair cost.

Thermals: Are There Real Heat Problems?

Yes, some users have reported overheating, throttling, freezes, spontaneous reboots, and shutdowns under sustained workloads. John Carmack’s early unit became the highest-profile example, and NVIDIA’s developer forum contains multiple stability reports.

Independent testing has produced mixed results:

LMSYS reported stable operation without thermal throttling in its tests.
StorageReview tested multiple GB10 systems and found substantial chassis differences. The Acer design peaked around 68°C GPU temperature, while several other units reached roughly 80–82°C under the same GPT-OSS-120B workload.
An ASUS Ascent GX10 review measured around 82°C GPU and 87°C CPU under sustained tests, with the NVMe remaining below 57°C. The system completed the workload without uncontrolled throttling.

Sources: StorageReview GB10 cooling comparison, ASUS Ascent GX10 review, and reporting on early Spark thermal and shutdown complaints.

The power specifications are often misreported. The external supply is 240 W, but NVIDIA specifies 140 W TDP for the GB10 SoC. The remaining budget covers the ConnectX-7 NIC, SSD, wireless hardware, USB-C, conversion losses, and other components. A system drawing around 100–150 W at the wall is not necessarily being capped to half of a supposed 240 W chip limit.

The fair assessment is:

The Founders Edition runs warm, some units and software combinations have shown real stability problems, and OEM cooling design matters. There is not enough evidence to say that all GB10 systems inevitably throttle or overheat.

Before keeping a new unit, I would:

Install current DGX OS, firmware, NIC firmware, and container tooling.
Run a sustained model-load, prefill, and decode stress test for several hours.
Monitor clocks, power, thermals, ECC or Xid errors, and system logs.
Test both the GPU and ConnectX link under load.
Leave clear airflow around the chassis and respect NVIDIA’s 30°C ideal ambient upper limit.
Buy from a vendor with a good return policy and support contract.
Prefer a better-cooled OEM chassis if sustained 24/7 serving is the goal.

CUDA Is the Main Reason to Buy GB10

Strix Halo and Apple Silicon can also provide 128 GB or more of unified memory. Neither provides the same CUDA environment.

DGX Spark is valuable as a small CUDA development box because it can run or approximate many production workflows:

PyTorch CUDA development
vLLM and SGLang serving
TensorRT-LLM and NVIDIA NIM containers
NCCL and distributed inference experiments
Blackwell FP4 and NVFP4 testing
Quantization and fine-tuning experiments
ComfyUI, diffusion, and CUDA-based image generation
Video-generation pipelines that have Arm64-compatible dependencies
Local OpenAI-compatible endpoints for application development

Code developed on Spark can be moved toward larger Blackwell infrastructure with fewer conceptual changes than code written only for Metal or a CPU backend. This is a real product feature.

CUDA Compatible Does Not Mean Frictionless

The system is Arm64, not x86-64, and its Blackwell target is sm_121. Launch-era software support was rough. Prebuilt wheels and containers sometimes omitted the architecture, custom CUDA extensions failed to compile, and vLLM configurations that worked on larger Blackwell GPUs did not automatically work on GB10.

Support has improved substantially by mid-2026. NVIDIA and vLLM now publish Spark-specific recipes and CUDA 13 images. The safe approach remains:

Use a container or wheel explicitly validated for sm_121.
Pin image digests or exact commits for working deployments.
Check model-specific recipes before assuming a quant format is accelerated.
Expect to rebuild custom CUDA extensions for Arm64.
Leave memory for JIT compilation; compiling kernels while 120 GB is allocated can freeze the system.

This is a first-gen developer platform. It is much better supported than it was at launch but not perfect.

Image and Video Generation

The 128 GB pool is useful for large diffusion and video models, multi-ControlNet graphs, multiple LoRAs, and workflows that would otherwise offload aggressively to system RAM.

CUDA is the advantage. Many image and video tools are written and optimised for NVIDIA first, so Spark can run workflows that are awkward or unsupported on Metal and ROCm.

The same caveat applies: speed. A high-end GeForce or RTX Pro card has much more memory bandwidth and can be substantially faster when the model fits in its VRAM. Spark is for when the workflow needs more than 32–48 GB, when an Arm64-compatible CUDA stack exists, or when you want one compact machine for LLM, image, video, and development work.

The single NVENC and NVDEC engines also make it less compelling as a high-throughput transcoding system.

Storage and OEM Systems

The NVIDIA Founders Edition ships with a 4 TB NVMe SSD. A few large FP8, BF16, NVFP4, GGUF, and diffusion checkpoints can consume terabytes.

Other GB10 systems may offer 1 TB, 2 TB, or 4 TB configurations. A 1 TB unit can be cheaper, but 1 TB could be restrictive. Check the exact chassis before assuming the SSD is easy to upgrade; several GB10 designs use a single compact M.2 slot rather than a standard desktop expansion layout.

Current pricing is inconsistent and changes with memory supply:

NVIDIA raised the 4 TB Founders Edition US price from $3,999 to $4,699 in February 2026.
ASUS lists the 1 TB Ascent GX10 from about $3,999, with paid 2 TB and 4 TB upgrades.
HP offers 1 TB and 4 TB configurations.
Dell and Lenovo GB10 systems are not always cheaper; channel pricing, support, storage, and warranty can make them more expensive than NVIDIA or ASUS.

Sources: ASUS Ascent GX10 store, NVIDIA price increase reporting, HP ZGX Nano, Dell Pro Max with GB10, and Lenovo ThinkStation PGX.

Do not choose purely by SSD size. Compare:

Cooling and sustained clocks
Warranty and business support
Noise
SSD replaceability
Included QSFP cable
Firmware update quality
Availability of local service
Actual delivered price after tax

DGX Spark vs AMD Strix Halo

AMD Strix Halo is the closest lower-cost alternative.

A Ryzen AI Max+ 395 platform can provide:

16 Zen 5 CPU cores and 32 threads
Radeon 8060S integrated graphics with 40 RDNA 3.5 compute units
Up to 128 GB LPDDR5X unified memory
256 GB/s memory bandwidth
Up to 120 W platform TDP
x86-64 compatibility
Windows or Linux
10 GbE on AMD’s developer platform

Official specifications: AMD Ryzen AI Halo developer platform.

The bandwidth is close enough that single-stream dense decode is often similar. Community comparisons using llama.cpp and vLLM have found near-identical token generation on some models, while Spark can be roughly 2–5x faster in prompt processing, particularly at longer context. Spark also has native FP4 Tensor Core paths and vastly stronger cluster networking.

Source: DGX Spark vs Strix Halo initial comparison.

Choose Strix Halo When

Price matters more than CUDA.
You mainly use llama.cpp, LM Studio, or GGUF models.
Dense single-user decode is the primary workload.
You want an x86 desktop that also handles normal applications and gaming.
10 GbE is enough.
You are comfortable with ROCm’s current model and framework coverage.

Choose GB10 When

CUDA is central to your development workflow.
You need vLLM, SGLang, TensorRT-LLM, NIM, or Blackwell FP4 paths.
Prompt processing and time to first token matter.
You want 200 Gb/s RoCE clustering.
You run large MoE models or concurrent serving.
You need image and video tools that are NVIDIA-first.
You want a development environment closer to NVIDIA production infrastructure.

For a home user who wants one large GGUF model in a chat UI, Strix Halo is often the better value. For a CUDA developer or a multi-node experiment, Spark earns its premium.

DGX Spark vs Mac Studio and M5 Max

As of July 2026, the current Mac Studio uses M4 Max or M3 Ultra. The M5 Max is currently available in MacBook Pro and supports up to 128 GB unified memory at up to 614 GB/s.

The important Apple bandwidth figures are:

Apple system	Maximum unified memory	Memory bandwidth
M4 Max Mac Studio	128 GB	546 GB/s
M5 Max MacBook Pro	128 GB	614 GB/s
M3 Ultra Mac Studio	512 GB	819 GB/s

Sources: Mac Studio technical specifications and M5 Max announcement.

For dense low-batch decode, Apple’s much higher memory bandwidth is a major advantage. A large-memory M3 Ultra can generate dense-model output far faster than Spark while also fitting enormous models.

Spark can still dominate prefill. EXO tested Llama 3.1 8B FP16 with an 8K prompt and found:

System / phase	Time
DGX Spark prefill	1.47 s
M3 Ultra prefill	5.57 s
DGX Spark generation	2.87 s
M3 Ultra generation	0.85 s

In that experiment, Spark was about 3.8x faster at prefill, while M3 Ultra was about 3.4x faster at generation. A hybrid setup using Spark for prefill and M3 Ultra for decode completed the task 2.8x faster than the Mac alone.

Source: EXO’s DGX Spark and Mac Studio prefill/decode experiment.

Mac advantages include high bandwidth, excellent power efficiency, quiet operation, large memory configurations, and a mature desktop experience. GB10 advantages include CUDA, native NVIDIA inference tooling, much faster networking, stronger Blackwell low-precision paths, and easier transfer of CUDA workflows to NVIDIA servers.

For pure local dense LLM chat, a high-memory Mac is often the better experience. For CUDA development, MoE inference, prefill-heavy agents, and clustering, GB10 is more capable.

Who Should Buy One?

DGX Spark or another GB10 system makes sense for:

CUDA developers who need a compact local Blackwell machine
Researchers testing models too large for 24–48 GB GPUs
People running 100–200B-class MoE models locally
Teams serving many concurrent local requests
Developers building multi-agent or bulk-processing pipelines
Users experimenting with two- to four-node distributed inference
Image and video developers who need more than consumer-GPU VRAM
Labs that value low power, small size, and quiet operation over maximum tokens per second

It is a poor buy for:

Someone who wants the fastest single-user dense-model chat
Anyone expecting RTX 5090-class inference speed because both are Blackwell
Users who only need 7B–32B models
Buyers who do not need CUDA or 200G networking
People unwilling to pin containers, troubleshoot Arm64 dependencies, or follow model-specific recipes
Anyone treating four-node clustering as a cheap and simple replacement for a server

Practical Buying Recommendations

For One Box

Buy the cheapest well-cooled GB10 system with the support and storage you need. A 1 TB model is acceptable when you already have fast network storage or plan to replace the SSD. For a self-contained model library, 2–4 TB is much more practical.

Use it with a well-supported NVFP4 or MXFP4 MoE model. Leave at least 8–16 GB of unified memory free, and more when compiling kernels or serving long contexts. Start with NVIDIA’s current recipe rather than a generic upstream command.

For Two Boxes

A direct QSFP connection is the sweet spot. You avoid an expensive switch and get enough memory for models such as DeepSeek V4 Flash. Budget for the cable, matching software versions, static network configuration, and time spent validating NCCL/RDMA.

For Four Boxes

Buy four only when you already know the target model and have a reproducible recipe. At roughly $16,000–$20,000 before networking and tax, the cluster competes with used servers, multi-GPU workstations, cloud rental, and higher-end Apple configurations.

Four Sparks can run astonishingly large models at low power. They do not provide better value or less operational work than conventional infrastructure.

Verdict

NVIDIA DGX Spark is both underrated and overmarketed.

It is underrated because single-stream dense-model reviews miss its strongest capabilities: 128 GB of coherent memory, CUDA, fast prefill, native FP4, useful batch throughput, 200 Gb/s networking, and the ability to cluster up to four tiny systems. Large MoE models such as gpt-oss-120B can run at practical speed on one box. DeepSeek V4 Flash can reach roughly 35–44 tok/s on two with a validated recipe, and custom paths can go faster. Four nodes can run GLM-5.2-class models locally.

It is overmarketed because 273 GB/s is slow for dense LLM decode, the petaflop figure assumes FP4 sparsity, the first-generation software stack has required careful pinning, some units have shown real thermal or stability problems. A dense 70B model at 2–5 tok/s is not a premium experience.

The core package is still unique:

128 GB unified memory. CUDA. Blackwell FP4. Strong prompt processing. 200 Gb/s RoCE. A 140 W SoC. Up to four-node clustering.

Buy it for large MoE models, concurrency, CUDA development, and distributed experimentation.

Do not buy it for fast dense-model decode.