Bonsai 2 27B
prism-ml/Ternary-Bonsai-2-27B-gguf
Prism ML · 1 recorded configuration
Measured setup
Hardware
Runtime and tool
Quantisation
MTP draft depth
Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.
1× RTX 3090 · llama.cpp
PQ2_0 · MTP off
PP · left axisTG · right axis
View exact measurements
| Context depth (tokens) | Fresh prefill PP (tok/s) | TG (tok/s) |
|---|---|---|
| 0 | 1,212.67 | 65.18 |
| 2,048 | 1,173.48 | 63.34 |
| 4,096 | 1,133.79 | 61.47 |
| 8,192 | 1,083.32 | 58.81 |
| 16,384 | 979.20 | 53.65 |
| 32,768 | 826.31 | 46.07 |
| 65,536 | 627.11 | 34.16 |
| 131,072 | 425.25 | 21.41 |
| 261,312 | 260.39 | 11.96 |
Concurrent-stream measurements
| Streams | Context depth (tokens) | Aggregate PP (tok/s) | Aggregate TG (tok/s) | TG / stream (tok/s) | Aggregate all (tok/s) |
|---|---|---|---|---|---|
| 2 | 8,192 | 868.28 | 86.99 | 43.49 | 220.44 |
| 3 | 8,192 | 854.22 | 89.78 | 29.93 | 241.13 |
| 4 | 8,192 | 821.02 | 89.91 | 22.48 | 246.89 |
| 5 | 8,192 | 767.57 | 96.77 | 19.35 | 268.92 |
| 6 | 8,192 | 744.00 | 93.75 | 15.62 | 263.55 |
| 7 | 8,192 | 733.81 | 92.95 | 13.28 | 263.77 |
| 8 | 8,192 | 727.00 | 97.66 | 12.21 | 278.32 |
Technical record and provenance
Source 1 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-context-20261016-off-c1
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- context
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 262,144 tokens
- Actual context per slot
- 262,144 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 0, 1 stream: PP mean 1,212.67 t/s (min 1,211.42, max 1,215.07, SD 1.70, n=3); TG mean 65.18 t/s (min 65.10, max 65.31, SD 0.09, n=3); All mean 175.24 t/s (min 175.04, max 175.57, SD 0.23, n=3) · KV 2,048, 1 stream: PP mean 1,173.48 t/s (min 1,168.84, max 1,175.83, SD 3.28, n=3); TG mean 63.34 t/s (min 63.26, max 63.39, SD 0.06, n=3); All mean 170.29 t/s (min 170.10, max 170.47, SD 0.15, n=3) · KV 4,096, 1 stream: PP mean 1,133.79 t/s (min 1,131.62, max 1,136.14, SD 1.85, n=3); TG mean 61.47 t/s (min 61.37, max 61.58, SD 0.09, n=3); All mean 165.15 t/s (min 164.90, max 165.48, SD 0.24, n=3) · KV 8,192, 1 stream: PP mean 1,083.32 t/s (min 1,079.15, max 1,086.45, SD 3.07, n=3); TG mean 58.81 t/s (min 58.76, max 58.90, SD 0.07, n=3); All mean 158.02 t/s (min 157.80, max 158.32, SD 0.22, n=3) · KV 16,384, 1 stream: PP mean 979.20 t/s (min 977.48, max 980.48, SD 1.26, n=3); TG mean 53.65 t/s (min 53.49, max 53.80, SD 0.13, n=3); All mean 144.03 t/s (min 143.59, max 144.45, SD 0.35, n=3) · KV 32,768, 1 stream: PP mean 826.31 t/s (min 824.54, max 829.44, SD 2.22, n=3); TG mean 46.07 t/s (min 45.97, max 46.16, SD 0.08, n=3); All mean 123.48 t/s (min 123.22, max 123.76, SD 0.22, n=3) · KV 65,536, 1 stream: PP mean 627.11 t/s (min 626.52, max 627.70, SD 0.48, n=3); TG mean 34.16 t/s (min 34.11, max 34.21, SD 0.04, n=3); All mean 91.80 t/s (min 91.68, max 91.96, SD 0.12, n=3) · KV 131,072, 1 stream: PP mean 425.25 t/s (min 424.82, max 425.64, SD 0.34, n=3); TG mean 21.41 t/s (min 21.39, max 21.43, SD 0.02, n=3); All mean 58.03 t/s (min 57.97, max 58.08, SD 0.05, n=3) · KV 261,312, 1 stream: PP mean 260.39 t/s (min 260.14, max 260.85, SD 0.33, n=3); TG mean 11.96 t/s (min 11.93, max 11.97, SD 0.02, n=3); All mean 32.70 t/s (min 32.62, max 32.74, SD 0.06, n=3)
- Measurement outcomes
- 0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured; 261,312 tokens · 1 stream: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-context-gpu0
Source 2 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c2
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 2 streams: PP mean 868.28 t/s (min 866.27, max 869.33, SD 1.42, n=3); TG mean 86.99 t/s (min 86.71, max 87.15, SD 0.19, n=3); All mean 220.44 t/s (min 219.88, max 220.74, SD 0.39, n=3)
- Measurement outcomes
- 8,192 tokens · 2 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0
Source 3 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c3
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 3 streams: PP mean 854.22 t/s (min 838.51, max 863.29, SD 11.15, n=3); TG mean 89.78 t/s (min 89.63, max 90.03, SD 0.18, n=3); All mean 241.13 t/s (min 240.11, max 241.72, SD 0.72, n=3)
- Measurement outcomes
- 8,192 tokens · 3 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0
Source 4 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c4
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 4 streams: PP mean 821.02 t/s (min 789.23, max 881.58, SD 42.84, n=3); TG mean 89.91 t/s (min 88.48, max 91.17, SD 1.11, n=3); All mean 246.89 t/s (min 242.44, max 251.82, SD 3.85, n=3)
- Measurement outcomes
- 8,192 tokens · 4 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0
Source 5 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c5
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 5 streams: PP mean 767.57 t/s (min 746.73, max 802.97, SD 25.16, n=3); TG mean 96.77 t/s (min 96.39, max 97.25, SD 0.36, n=3); All mean 268.92 t/s (min 267.15, max 271.73, SD 2.01, n=3)
- Measurement outcomes
- 8,192 tokens · 5 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0
Source 6 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c6
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 6 streams: PP mean 744.00 t/s (min 741.79, max 745.75, SD 1.65, n=3); TG mean 93.75 t/s (min 93.68, max 93.80, SD 0.05, n=3); All mean 263.55 t/s (min 263.34, max 263.70, SD 0.15, n=3)
- Measurement outcomes
- 8,192 tokens · 6 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0
Source 7 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c7
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 7 streams: PP mean 733.81 t/s (min 733.20, max 734.16, SD 0.43, n=3); TG mean 92.95 t/s (min 92.91, max 92.98, SD 0.03, n=3); All mean 263.77 t/s (min 263.65, max 263.85, SD 0.09, n=3)
- Measurement outcomes
- 8,192 tokens · 7 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0
Source 8 · 2026-10-16
- Source run ID
- pub-20261016-trpro-bonsai-pq2-prism-concurrency-20261016-off-c8
- Page entry ID
- bonsai-2-27b-pq2-0-single-gpu-20261016-off-45e9e6a509
- Entry sweep
- concurrency
- Raw entry label
- Bonsai 2 27B PQ2_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-16
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.2.0-dev (build 10777, commit e3de71522) built with GNU 11.4.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 8 streams: PP mean 727.00 t/s (min 726.16, max 727.73, SD 0.65, n=3); TG mean 97.66 t/s (min 97.60, max 97.73, SD 0.06, n=3); All mean 278.32 t/s (min 278.17, max 278.53, SD 0.15, n=3)
- Measurement outcomes
- 8,192 tokens · 8 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting. PrismML CUDA fork; MTP1–3 unavailable in the pinned checkpoint. Original stock-engine PQ2 incompatibility is retained separately. Long-context slowdown cause has not been established.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-16-bonsai-prism/campaign-run/bonsai-pq2-prism-concurrency-gpu0