Qwen3.8 27B
Qwen/Qwen3.8-27B
Qwen · 22 recorded configurations
Measured setup
Hardware
Runtime and tool
Quantisation
MTP draft depth
Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.
1× RTX 3090 · ExLlamaV3
EXL3 4.00bpw · MTP off
PP · left axisTG · right axis
View exact measurements
| Context depth (tokens) | Fresh prefill PP (tok/s) | TG (tok/s) |
|---|---|---|
| 0 | 932.53 | 36.52 |
| 16,384 | 768.44 | 25.46 |
| 204,032 | 258.56 | 19.27 |
Concurrent-stream measurements
| Streams | Context depth (tokens) | Aggregate PP (tok/s) | Aggregate TG (tok/s) | TG / stream (tok/s) | Aggregate all (tok/s) |
|---|---|---|---|---|---|
| 2 | 8,192 | 905.66 | 64.79 | 32.39 | 170.04 |
| 3 | 8,192 | 936.20 | 86.70 | 28.90 | 219.45 |
| 4 | 8,192 | 951.37 | 108.96 | 27.24 | 265.96 |
| 5 | 8,192 | 953.45 | 129.35 | 25.87 | 305.23 |
| 6 | 8,192 | 940.54 | 145.40 | 24.23 | 333.18 |
| 7 | 8,192 | 937.33 | 159.91 | 22.84 | 357.68 |
| 8 | 8,192 | 942.76 | 179.10 | 22.39 | 389.36 |
Technical record and provenance
Source 1 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-context-fit-1-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- context
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · context-fit 204800-token allocation
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 204,800 tokens
- Actual context per slot
- 204,800 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 0, 1 stream: PP mean 932.53 t/s (min 928.82, max 934.72, SD 3.24, n=3); TG mean 36.52 t/s (min 36.49, max 36.56, SD 0.03, n=3); All mean 101.61 t/s (min 101.51, max 101.71, SD 0.10, n=3) · KV 16,384, 1 stream: PP mean 768.44 t/s (min 767.40, max 770.24, SD 1.56, n=3); TG mean 25.46 t/s (min 25.45, max 25.46, SD 0.01, n=3); All mean 71.62 t/s (min 71.60, max 71.64, SD 0.02, n=3) · KV 204,032, 1 stream: PP mean 258.56 t/s (min 258.18, max 258.79, SD 0.33, n=3); TG mean 19.27 t/s (min 19.27, max 19.27, SD 0.00, n=3); All mean 50.32 t/s (min 50.31, max 50.33, SD 0.01, n=3)
- Measurement outcomes
- 0 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 204,032 tokens · 1 stream: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · context-fit 204800-token allocation. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-context-off-c1; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f97687218e995c9d50c2ea704b67dbe81c392cb1ead1e10a53e2f4d2b09c880f. Separately allocated 204800 tokens per stream for this MTP setting. The allocation search advances in 8192-token steps; its recorded fitted capacity is not a token-exact VRAM maximum. The selected boundary and the 0/16K samples are retained from their recorded source attempts.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-exl-context-fit/4bpw-gpu0/runs/off-204800-boundary/4.00bpw/context/off-c1
Source 2 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-2-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 2, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 2 streams: PP mean 905.66 t/s (min 903.71, max 906.74, SD 1.69, n=3); TG mean 64.79 t/s (min 64.66, max 64.89, SD 0.12, n=3); All mean 170.04 t/s (min 169.69, max 170.29, SD 0.31, n=3)
- Measurement outcomes
- 8,192 tokens · 2 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 2, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c2; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c2
Source 3 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-3-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 3, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 3 streams: PP mean 936.20 t/s (min 934.45, max 939.00, SD 2.45, n=3); TG mean 86.70 t/s (min 86.63, max 86.80, SD 0.09, n=3); All mean 219.45 t/s (min 219.23, max 219.77, SD 0.29, n=3)
- Measurement outcomes
- 8,192 tokens · 3 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 3, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c3; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c3
Source 4 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-4-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 4, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 4 streams: PP mean 951.37 t/s (min 949.83, max 953.14, SD 1.67, n=3); TG mean 108.96 t/s (min 108.89, max 109.08, SD 0.10, n=3); All mean 265.96 t/s (min 265.78, max 266.29, SD 0.29, n=3)
- Measurement outcomes
- 8,192 tokens · 4 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 4, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c4; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c4
Source 5 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-5-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 5, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 5 streams: PP mean 953.45 t/s (min 952.98, max 953.83, SD 0.43, n=3); TG mean 129.35 t/s (min 129.35, max 129.35, SD 0.00, n=3); All mean 305.23 t/s (min 305.20, max 305.26, SD 0.03, n=3)
- Measurement outcomes
- 8,192 tokens · 5 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 5, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c5; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c5
Source 6 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-6-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 6, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 6 streams: PP mean 940.54 t/s (min 938.89, max 942.32, SD 1.72, n=3); TG mean 145.40 t/s (min 145.34, max 145.44, SD 0.05, n=3); All mean 333.18 t/s (min 333.11, max 333.22, SD 0.06, n=3)
- Measurement outcomes
- 8,192 tokens · 6 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 6, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c6; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c6
Source 7 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-7-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 7, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 7 streams: PP mean 937.33 t/s (min 936.23, max 939.29, SD 1.70, n=3); TG mean 159.91 t/s (min 159.85, max 159.94, SD 0.05, n=3); All mean 357.68 t/s (min 357.62, max 357.78, SD 0.09, n=3)
- Measurement outcomes
- 8,192 tokens · 7 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 7, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c7; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c7
Source 8 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-4bpw-gpu0-4bpw-off-concurrency-8-20261015
- Page entry ID
- trpro-exl-qwen38-27b-4bpw-gpu0-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 4.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 8, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 8 streams: PP mean 942.76 t/s (min 937.31, max 948.69, SD 5.70, n=3); TG mean 179.10 t/s (min 178.86, max 179.53, SD 0.38, n=3); All mean 389.36 t/s (min 388.36, max 390.71, SD 1.22, n=3)
- Measurement outcomes
- 8,192 tokens · 8 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 4.00bpw · MTP off · concurrency 8, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 0 (GPU-7aa53e24-dc9d-bf8d-96cc-ef8c05c66dab) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 400bpw-concurrency-off-c8; model revision 113cf7ab958054860e43fb7f3063b1af19171095; source manifest SHA-256 f4213ce466cd62992d252fcd14c623cb23cee191ed865703639e884f2df15d01. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-4bpw-gpu0-independent-gpus-validation-20261015/4.00bpw/concurrency/off-c8