Qwen3.8 Flash
Qwen/Qwen3.8-Flash-Next
Qwen · 2 recorded configurations
Measured setup
Hardware
Runtime and tool
Quantisation
MTP draft depth
Recorded setup
Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.
2× RTX 3090 · ExLlamaV3
EXL3 4.05bpw_h6_ng6 · MTP 2
PP · left axisTG · right axis
View exact measurements
| Context depth (tokens) | Bulk prefix PP (tok/s) | Fresh prefill PP (tok/s) | TG (tok/s) |
|---|---|---|---|
| 0 | — | 338.65 | 87.11 |
| 2,048 | 718.39 | 257.57 | 92.23 |
| 4,096 | 1,041.75 | 287.52 | 89.28 |
| 8,192 | 1,436.73 | 237.07 | 89.32 |
| 16,384 | 1,446.60 | 280.35 | 95.46 |
| 32,768 | 1,446.76 | 226.16 | 93.14 |
| 65,536 | 1,434.64 | 228.25 | 90.39 |
| 131,072 | 1,419.74 | 266.96 | 89.90 |
| 261,376 | 1,391.06 | 315.56 | 81.53 |
Technical record and provenance
Run details
- Source run ID
- rtx3090x2-qwen3.8-flash-exl3-mtp2-fixed262144-2026-09-27
- Configuration label
- 288 CPU / 224 GPU · fixed 262144 allocation
- Measured date
- 2026-09-27
- Hardware
- 2× RTX 3090
- Runtime
- exllamav3 · ExLlamaV3 1.5.0+cu128.torch2.9.0; installed source/binary hashes in evidence
- Weight placement
- partially-offloaded
- Placement detail
- 288 CPU / 224 GPU experts per target layer; static decode-trained profile. Full-GPU MTP2 draft on CUDA 1, all 512 draft experts; n-gram table in RAM.
- KV-cache precision
- EXL K8/V8 with QSA side planes and recurrent state
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- MTP · draft max 2
- Measured repetitions
- 3
- Notes
- One successful model load, same PID/model/cache identities across all 36 iterations. Target 288 CPU / 224 GPU experts per layer; fixed target and draft capacity 262144 throughout, 4.789 GiB allocated GPU cache tensors including QSA/recurrent state (not whole-process VRAM). Cache/history/recurrent state reset each iteration without unloading weights. Largest existing prefix 261376 + PP512 + TG256 = native total limit 262144; no RoPE override and no full 262144-token prefix measurement. Initial fixed262144 allocation at 256 CPU experts failed native autosplit before loading the draft; user authorized more CPU experts. Maximum-prefix build averages 187.898 s; prefix-to-completion averages 192.663 s, excluding loading and reset. C1; existing prefix + fresh PP512 + TG256; one warm-up and three measured repetitions per depth, arithmetic mean of per-repetition rates. Bulk prefix is rebuilt in chunks up to 8192 tokens and timed separately: bulkPrefillTps = mean(prefix tokens * 1000 / prefixMs), not fresh PP512. Synchronized host timing includes final-position logits and the draft prompt cache in PP; TG includes first sampling from PP logits and native MTP draft/verify to exactly 256 outputs (EOS ignored). Load, reset, prefix build and serialization are excluded from fresh PP/TG; post-decode queue drain is excluded from TG. Frozen decode-trained expert profile SHA256 6ac003f37ca2dcb4e56b15fa72d0d5726e8722cd2a7e6ee3d3f73b0d406b2b34; static hot-expert placement, no dynamic swapping. Target has 48 routed layers, 512 experts per layer, top 10 selected per token; CPU threads 12, slots 4. Native MTP2 draft stays on CUDA 1 with all 512 experts and an independent GPU embedding. N-gram table in RAM (trellis_ram); vision was never loaded, not CPU-offloaded. EXL K8/V8 includes QSA side planes and recurrent state, not llama.cpp Q8_0. Model revision 55a732e0c4c3d4614bc42b68493bb930d9b02c0a. Corpus content changes with depth; not an isolated context-length microbenchmark or quality-equivalence proof. Native MTP stream-tap semantics remain an upstream caveat.
- Source article
- /local-ai/qwen3-8-flash-local-ai/
- Raw log path
research/benchmarks/logs/trpro/2026-09-27-exl-context-sweep/publication-evidence.json