Qwen3.5 122B-A10B

Qwen/Qwen3.5-122B-A10B

10B active / 122B totalMoEText generationImage → textText inputImage input

Qwen · 1 recorded configuration

Measured setup

Hardware

2× RTX 3090 · CPU offload

Runtime and tool

llama-bench

Quantisation

UD-Q4_K_S

MTP draft depth

MTP off

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

PP · left axisTG · right axis

View exact measurements

Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Qwen3.5 122B-A10B
Context depth (tokens)	Fresh prefill PP (tok/s)	TG (tok/s)
0	312.40	31.11
2,048	306.30	30.90
4,096	298.80	30.75
8,192	294.70	30.35
16,384	289.90	29.32
32,768	279.60	27.36

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Qwen3.5 122B-A10B
Streams	Aggregate PP (tok/s)	Aggregate TG (tok/s)	TG / stream (tok/s)	Aggregate all (tok/s)
1	312.40	31.11	—	—
1	255.60	35.29	35.29	82.96
2	256.70	35.25	17.63	82.97
4	259.90	49.97	12.49	108.22

Technical record and provenance

Run details

Source run ID: rtx3090x2-qwen3.5-122b-a10b-ud-q4ks-2026-08
Hardware: 2× RTX 3090
Runtime: llamaCpp · llama-bench fc6545d (build 50)
Weight placement: partially-offloaded
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 4,096
Microbatch size: 256
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: None (standard decoding)
Measured repetitions: 3
Notes: UD-Q4_K_S, batch 4096/256. Context sweep: automatic fit with 2,048 MiB free per GPU (partially offloaded). Concurrency: 20 MoE expert layers on CPU, remaining layers split 65:35 (aggregate rates). The model is in the calculator catalog, but because this run is partially offloaded it does not feed the calculator's fully-resident calibration path.
Source article: /local-ai/rtx-3090-for-local-ai-inference/

Compare measured entries