Qwen3.6 35B-A3B

Qwen/Qwen3.6-35B-A3B

3B active / 35B totalMoEText generationImage → textText inputImage input

Qwen · 9 recorded configurations

Measured setup

Hardware

  • 1× RTX 3090 · GPU-resident

Runtime and tool

  • live llama-server

Quantisation

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

1× RTX 3090 · llama.cpp

UD-IQ4_XS · MTP 3

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Qwen3.6 35B-A3B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
02,617.11240.66
2,0482,154.94235.88
4,0962,113.03233.28
8,1922,027.09229.13
16,3841,727.84220.40
32,7681,387.81204.44
65,5361,000.34178.75
131,072620.61141.80
261,312369.38102.58

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Qwen3.6 35B-A3B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
28,1921,848.41291.56145.78675.88
38,1921,692.74324.80108.27778.30
48,1921,603.63336.0484.01843.14
58,1921,580.96346.3369.27894.96
68,1921,557.77353.6558.94932.53
78,1921,512.62355.1650.74948.25
88,1921,525.36358.6944.84974.26
Technical record and provenance

Source 1 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-context-2026-10-14-single-gpu-isolated-mtp-3-c1
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
context
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
262,144 tokens
Actual context per slot
262,144 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 0, 1 stream: PP mean 2,617.11 t/s (min 2,597.92, max 2,632.77, SD 14.45, n=3); TG mean 240.66 t/s (min 239.41, max 241.69, SD 0.94, n=3); All mean 594.64 t/s (min 592.82, max 595.84, SD 1.31, n=3) · KV 2,048, 1 stream: PP mean 2,154.94 t/s (min 2,124.17, max 2,195.89, SD 30.15, n=3); TG mean 235.88 t/s (min 234.35, max 236.95, SD 1.11, n=3); All mean 566.36 t/s (min 562.82, max 569.29, SD 2.68, n=3) · KV 4,096, 1 stream: PP mean 2,113.03 t/s (min 2,054.92, max 2,151.48, SD 41.80, n=3); TG mean 233.28 t/s (min 229.73, max 235.20, SD 2.51, n=3); All mean 559.17 t/s (min 549.53, max 564.77, SD 6.84, n=3) · KV 8,192, 1 stream: PP mean 2,027.09 t/s (min 1,986.43, max 2,056.98, SD 29.79, n=3); TG mean 229.13 t/s (min 228.58, max 230.06, SD 0.66, n=3); All mean 546.74 t/s (min 546.39, max 547.29, SD 0.40, n=3) · KV 16,384, 1 stream: PP mean 1,727.84 t/s (min 1,712.04, max 1,737.17, SD 11.23, n=3); TG mean 220.40 t/s (min 219.96, max 221.09, SD 0.49, n=3); All mean 513.33 t/s (min 512.25, max 515.07, SD 1.24, n=3) · KV 32,768, 1 stream: PP mean 1,387.81 t/s (min 1,375.28, max 1,399.92, SD 10.06, n=3); TG mean 204.44 t/s (min 204.14, max 204.72, SD 0.23, n=3); All mean 461.34 t/s (min 460.14, max 462.85, SD 1.13, n=3) · KV 65,536, 1 stream: PP mean 1,000.34 t/s (min 995.92, max 1,003.82, SD 3.29, n=3); TG mean 178.75 t/s (min 178.20, max 179.43, SD 0.51, n=3); All mean 384.42 t/s (min 383.62, max 385.88, SD 1.03, n=3) · KV 131,072, 1 stream: PP mean 620.61 t/s (min 591.89, max 635.81, SD 20.32, n=3); TG mean 141.80 t/s (min 140.82, max 142.44, SD 0.70, n=3); All mean 284.09 t/s (min 279.04, max 286.71, SD 3.57, n=3) · KV 261,312, 1 stream: PP mean 369.38 t/s (min 362.56, max 373.32, SD 4.84, n=3); TG mean 102.58 t/s (min 102.42, max 102.69, SD 0.12, n=3); All mean 192.39 t/s (min 190.58, max 193.54, SD 1.30, n=3)
Measurement outcomes
0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured; 261,312 tokens · 1 stream: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-context-isolated

Source 2 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c2
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 2 streams: PP mean 1,848.41 t/s (min 1,683.52, max 1,955.89, SD 118.37, n=3); TG mean 291.56 t/s (min 290.42, max 293.17, SD 1.17, n=3); All mean 675.88 t/s (min 661.20, max 683.81, SD 10.40, n=3)
Measurement outcomes
8,192 tokens · 2 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Source 3 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c3
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 3 streams: PP mean 1,692.74 t/s (min 1,635.11, max 1,754.69, SD 48.91, n=3); TG mean 324.80 t/s (min 323.07, max 326.13, SD 1.28, n=3); All mean 778.30 t/s (min 769.40, max 786.55, SD 7.02, n=3)
Measurement outcomes
8,192 tokens · 3 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Source 4 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c4
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 4 streams: PP mean 1,603.63 t/s (min 1,588.78, max 1,614.61, SD 10.89, n=3); TG mean 336.04 t/s (min 334.61, max 337.12, SD 1.05, n=3); All mean 843.14 t/s (min 840.47, max 848.09, SD 3.50, n=3)
Measurement outcomes
8,192 tokens · 4 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Source 5 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c5
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 5 streams: PP mean 1,580.96 t/s (min 1,555.61, max 1,615.68, SD 25.40, n=3); TG mean 346.33 t/s (min 344.89, max 347.12, SD 1.02, n=3); All mean 894.96 t/s (min 889.00, max 902.28, SD 5.51, n=3)
Measurement outcomes
8,192 tokens · 5 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Source 6 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c6
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 6 streams: PP mean 1,557.77 t/s (min 1,536.13, max 1,569.18, SD 15.31, n=3); TG mean 353.65 t/s (min 351.83, max 354.87, SD 1.31, n=3); All mean 932.53 t/s (min 925.93, max 936.89, SD 4.75, n=3)
Measurement outcomes
8,192 tokens · 6 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Source 7 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c7
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 7 streams: PP mean 1,512.62 t/s (min 1,497.44, max 1,522.99, SD 10.97, n=3); TG mean 355.16 t/s (min 353.28, max 356.73, SD 1.43, n=3); All mean 948.25 t/s (min 944.23, max 950.41, SD 2.85, n=3)
Measurement outcomes
8,192 tokens · 7 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Source 8 · 2026-10-15

Source run ID
pub-20261016-trpro-qwen36-iq4-xs-concurrency-2026-10-14-single-gpu-isolated-mtp-3-c8
Page entry ID
qwen3-6-35b-a3b-ud-iq4-xs-single-gpu-20261016-mtp-3-041246436c
Entry sweep
concurrency
Raw entry label
Qwen3.6 35B-A3B UD-IQ4_XS — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 8 streams: PP mean 1,525.36 t/s (min 1,494.37, max 1,556.01, SD 25.16, n=3); TG mean 358.69 t/s (min 356.53, max 360.77, SD 1.73, n=3); All mean 974.26 t/s (min 964.83, max 983.34, SD 7.56, n=3)
Measurement outcomes
8,192 tokens · 8 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-14-approved-models-single-gpu/qwen36-iq4-xs-concurrency-isolated

Compare measured entries