Qwen3.8 27B
Qwen/Qwen3.8-27B
Qwen · 22 recorded configurations
Measured setup
Hardware
Runtime and tool
Quantisation
MTP draft depth
Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.
1× RTX 3090 · ExLlamaV3
EXL3 5.00bpw · MTP off
PP · left axisTG · right axis
View exact measurements
| Context depth (tokens) | Fresh prefill PP (tok/s) | TG (tok/s) |
|---|---|---|
| 0 | 897.60 | 28.15 |
| 16,384 | 743.27 | 23.77 |
| 130,304 | 335.04 | 19.61 |
Concurrent-stream measurements
| Streams | Context depth (tokens) | Aggregate PP (tok/s) | Aggregate TG (tok/s) | TG / stream (tok/s) | Aggregate all (tok/s) |
|---|---|---|---|---|---|
| 2 | 8,192 | 875.63 | 49.25 | 24.63 | 132.81 |
| 3 | 8,192 | 900.42 | 72.36 | 24.12 | 187.01 |
| 4 | 8,192 | 925.41 | 92.20 | 23.05 | 230.64 |
| 5 | 8,192 | 931.94 | 108.96 | 21.79 | 264.93 |
| 6 | 8,192 | 915.68 | 124.10 | 20.68 | 292.90 |
| 7 | 8,192 | 919.65 | 137.14 | 19.59 | 316.90 |
| 8 | 8,192 | 915.16 | 153.36 | 19.17 | 344.58 |
Technical record and provenance
Source 1 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-context-fit-1-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- context
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · context-fit 131072-token allocation
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 131,072 tokens
- Actual context per slot
- 131,072 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 0, 1 stream: PP mean 897.60 t/s (min 895.31, max 901.87, SD 3.70, n=3); TG mean 28.15 t/s (min 28.04, max 28.35, SD 0.17, n=3); All mean 79.46 t/s (min 79.19, max 79.97, SD 0.45, n=3) · KV 16,384, 1 stream: PP mean 743.27 t/s (min 741.45, max 745.57, SD 2.10, n=3); TG mean 23.77 t/s (min 23.71, max 23.84, SD 0.07, n=3); All mean 67.01 t/s (min 66.85, max 67.23, SD 0.20, n=3) · KV 130,304, 1 stream: PP mean 335.04 t/s (min 334.98, max 335.10, SD 0.06, n=3); TG mean 19.61 t/s (min 19.61, max 19.62, SD 0.01, n=3); All mean 52.67 t/s (min 52.66, max 52.69, SD 0.02, n=3)
- Measurement outcomes
- 0 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 130,304 tokens · 1 stream: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · context-fit 131072-token allocation. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-context-off-c1; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 3ac3ebed631636c039113a3b29ace36523dbad9484e5a5af31e3f21cd5114457. Separately allocated 131072 tokens per stream for this MTP setting. The allocation search advances in 8192-token steps; its recorded fitted capacity is not a token-exact VRAM maximum. The selected boundary and the 0/16K samples are retained from their recorded source attempts.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-exl-context-fit/5bpw-gpu1/runs/off-131072-boundary/5.00bpw/context/off-c1
Source 2 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-2-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 2, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 2 streams: PP mean 875.63 t/s (min 874.61, max 877.32, SD 1.48, n=3); TG mean 49.25 t/s (min 49.08, max 49.38, SD 0.16, n=3); All mean 132.81 t/s (min 132.38, max 133.15, SD 0.39, n=3)
- Measurement outcomes
- 8,192 tokens · 2 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 2, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c2; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c2
Source 3 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-3-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 3, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 3 streams: PP mean 900.42 t/s (min 898.02, max 901.92, SD 2.10, n=3); TG mean 72.36 t/s (min 72.22, max 72.50, SD 0.14, n=3); All mean 187.01 t/s (min 186.65, max 187.37, SD 0.36, n=3)
- Measurement outcomes
- 8,192 tokens · 3 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 3, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c3; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c3
Source 4 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-4-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 4, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 4 streams: PP mean 925.41 t/s (min 923.68, max 927.42, SD 1.89, n=3); TG mean 92.20 t/s (min 92.09, max 92.29, SD 0.10, n=3); All mean 230.64 t/s (min 230.34, max 230.92, SD 0.29, n=3)
- Measurement outcomes
- 8,192 tokens · 4 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 4, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c4; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c4
Source 5 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-5-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 5, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 5 streams: PP mean 931.94 t/s (min 930.75, max 933.70, SD 1.56, n=3); TG mean 108.96 t/s (min 108.38, max 109.38, SD 0.52, n=3); All mean 264.93 t/s (min 263.75, max 265.85, SD 1.08, n=3)
- Measurement outcomes
- 8,192 tokens · 5 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 5, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c5; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c5
Source 6 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-6-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 6, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 6 streams: PP mean 915.68 t/s (min 914.54, max 917.32, SD 1.45, n=3); TG mean 124.10 t/s (min 123.95, max 124.20, SD 0.13, n=3); All mean 292.90 t/s (min 292.55, max 293.10, SD 0.30, n=3)
- Measurement outcomes
- 8,192 tokens · 6 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 6, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c6; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c6
Source 7 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-7-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 7, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 7 streams: PP mean 919.65 t/s (min 917.11, max 923.27, SD 3.22, n=3); TG mean 137.14 t/s (min 136.63, max 137.95, SD 0.72, n=3); All mean 316.90 t/s (min 315.79, max 318.64, SD 1.53, n=3)
- Measurement outcomes
- 8,192 tokens · 7 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 7, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c7; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c7
Source 8 · 2026-10-15
- Source run ID
- trpro-exl-qwen38-27b-5bpw-gpu1-5bpw-off-concurrency-8-20261015
- Page entry ID
- trpro-exl-qwen38-27b-5bpw-gpu1-off
- Entry sweep
- concurrency
- Raw entry label
- Qwen3.8 27B EXL3 5.00bpw · MTP off
- Configuration label
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 8, 8192-token prefix, 8960-token allocation per stream
- Measured date
- 2026-10-15
- Hardware
- 1× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; native extension SHA-256 c1d02b410ef729618077ba287f01f05b229f31d118a7aae04a1b7e56142e3b2e
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- One physical RTX 3090; target model and native MTP compute tensors were observed on CUDA0, with no CPU expert offload or model split.
- KV-cache precision
- EXL native K8/V8 with QSA side planes and recurrent state
- Requested context per slot
- 8,960 tokens
- Actual context per slot
- 8,960 tokens
- Native model context
- 262,144 tokens
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- ExLlamaV3 persistent worker time.perf_counter_ns with torch.cuda.synchronize at phase boundaries; fresh PP excludes prefix preparation; aggregate TG is across streams
- Throughput spreads
- KV 8,192, 8 streams: PP mean 915.16 t/s (min 914.17, max 916.94, SD 1.55, n=3); TG mean 153.36 t/s (min 152.79, max 154.43, SD 0.93, n=3); All mean 344.58 t/s (min 343.55, max 346.56, SD 1.71, n=3)
- Measurement outcomes
- 8,192 tokens · 8 streams: measured
- Notes
- 1× RTX3090 — no splitting · EXL3 5.00bpw · MTP off · concurrency 8, 8192-token prefix, 8960-token allocation per stream. Persistent EXL native worker: 512 fresh prompt tokens and 256 generated tokens; one warmup then three measured repetitions. PP is fresh-prompt processing after the populated prefix; TG is aggregate decode across concurrent streams. Throughput means and sample standard deviations are recomputed from the retained raw repetitions. EXL native K8/V8 cache storage includes QSA side planes and recurrent state; this is not GGUF Q8_0. Physical GPU 1 (GPU-0a2bed40-39e4-e67f-5234-868704f9704b) was selected as the single device. Runtime evidence maps that physical GPU to one logical CUDA0. Source configuration 500bpw-concurrency-off-c8; model revision a35e75a73baee51da709329d19294245cbeeb5d8; source manifest SHA-256 f889b720ea21d731842579e93d41eb5e4ed995a63475a7e69874a2330b529192. The concurrency source used an 8192-token prefix and a fixed 8960-token per-stream cache allocation; this allocation is independent of context-fit capacities.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/trpro-exl-qwen38-27b-5bpw-gpu1-independent-gpus-validation-20261015/5.00bpw/concurrency/off-c8