Gemma 4 12B
google/gemma-4-12B-it
Google · 4 recorded configurations
Measured setup
Hardware
Runtime and tool
Quantisation
MTP draft depth
Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.
1× RTX 3090 · llama.cpp
Q8_0 · MTP off
PP · left axisTG · right axis
View exact measurements
| Context depth (tokens) | Fresh prefill PP (tok/s) | TG (tok/s) |
|---|---|---|
| 0 | 2,399.15 | 51.88 |
| 2,048 | 1,470.38 | 50.45 |
| 4,096 | 1,460.56 | 49.78 |
| 8,192 | 1,382.17 | 49.45 |
| 16,384 | 1,317.09 | 48.48 |
| 32,768 | 1,120.05 | 46.34 |
| 65,536 | 911.80 | 42.56 |
| 131,072 | 647.61 | 37.09 |
| 261,312 | 415.21 | 29.39 |
Concurrent-stream measurements
| Streams | Context depth (tokens) | Aggregate PP (tok/s) | Aggregate TG (tok/s) | TG / stream (tok/s) | Aggregate all (tok/s) |
|---|---|---|---|---|---|
| 2 | 8,192 | 1,107.55 | 87.28 | 43.64 | 229.68 |
| 3 | 8,192 | 980.61 | 107.03 | 35.68 | 282.12 |
| 4 | 8,192 | 947.74 | 118.56 | 29.64 | 319.15 |
| 5 | 8,192 | 891.46 | 130.54 | 26.11 | 356.04 |
| 6 | 8,192 | 883.00 | 134.39 | 22.40 | 371.29 |
| 7 | 8,192 | 895.78 | 134.38 | 19.20 | 377.62 |
| 8 | 8,192 | 904.33 | 140.53 | 17.57 | 398.80 |
Technical record and provenance
Source 1 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-context-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c1
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- context
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 262,144 tokens
- Actual context per slot
- 262,144 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 0, 1 stream: PP mean 2,399.15 t/s (min 2,391.53, max 2,404.31, SD 5.50, n=3); TG mean 51.88 t/s (min 51.83, max 51.94, SD 0.05, n=3); All mean 148.24 t/s (min 148.13, max 148.40, SD 0.11, n=3) · KV 2,048, 1 stream: PP mean 1,470.38 t/s (min 1,443.73, max 1,484.05, SD 18.85, n=3); TG mean 50.45 t/s (min 50.36, max 50.55, SD 0.08, n=3); All mean 140.76 t/s (min 140.40, max 141.12, SD 0.29, n=3) · KV 4,096, 1 stream: PP mean 1,460.56 t/s (min 1,441.20, max 1,474.34, SD 14.09, n=3); TG mean 49.78 t/s (min 49.71, max 49.91, SD 0.09, n=3); All mean 138.94 t/s (min 138.68, max 139.35, SD 0.29, n=3) · KV 8,192, 1 stream: PP mean 1,382.17 t/s (min 1,348.23, max 1,401.80, SD 24.10, n=3); TG mean 49.45 t/s (min 49.44, max 49.48, SD 0.02, n=3); All mean 137.56 t/s (min 137.28, max 137.70, SD 0.20, n=3) · KV 16,384, 1 stream: PP mean 1,317.09 t/s (min 1,313.50, max 1,321.87, SD 3.52, n=3); TG mean 48.48 t/s (min 48.33, max 48.58, SD 0.11, n=3); All mean 134.59 t/s (min 134.18, max 134.91, SD 0.31, n=3) · KV 32,768, 1 stream: PP mean 1,120.05 t/s (min 1,091.21, max 1,135.07, SD 20.40, n=3); TG mean 46.34 t/s (min 46.25, max 46.42, SD 0.07, n=3); All mean 127.46 t/s (min 126.99, max 127.78, SD 0.34, n=3) · KV 65,536, 1 stream: PP mean 911.80 t/s (min 906.08, max 917.11, SD 4.51, n=3); TG mean 42.56 t/s (min 42.52, max 42.63, SD 0.05, n=3); All mean 115.85 t/s (min 115.71, max 116.09, SD 0.17, n=3) · KV 131,072, 1 stream: PP mean 647.61 t/s (min 645.38, max 649.69, SD 1.76, n=3); TG mean 37.09 t/s (min 37.04, max 37.12, SD 0.04, n=3); All mean 98.91 t/s (min 98.82, max 99.04, SD 0.09, n=3) · KV 261,312, 1 stream: PP mean 415.21 t/s (min 411.68, max 417.04, SD 2.50, n=3); TG mean 29.39 t/s (min 29.34, max 29.42, SD 0.03, n=3); All mean 76.34 t/s (min 76.23, max 76.44, SD 0.09, n=3)
- Measurement outcomes
- 0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured; 261,312 tokens · 1 stream: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-gpu0-overlap-validation/gemma12-context-gpu0-validation
Source 2 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c2
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 2 streams: PP mean 1,107.55 t/s (min 1,051.42, max 1,210.19, SD 72.68, n=3); TG mean 87.28 t/s (min 87.07, max 87.58, SD 0.22, n=3); All mean 229.68 t/s (min 227.87, max 232.08, SD 1.77, n=3)
- Measurement outcomes
- 8,192 tokens · 2 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0
Source 3 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c3
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 3 streams: PP mean 980.61 t/s (min 950.43, max 996.38, SD 21.35, n=3); TG mean 107.03 t/s (min 104.45, max 108.80, SD 1.87, n=3); All mean 282.12 t/s (min 274.84, max 286.77, SD 5.22, n=3)
- Measurement outcomes
- 8,192 tokens · 3 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0
Source 4 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c4
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 4 streams: PP mean 947.74 t/s (min 935.65, max 956.46, SD 8.83, n=3); TG mean 118.56 t/s (min 118.35, max 118.71, SD 0.15, n=3); All mean 319.15 t/s (min 318.42, max 320.00, SD 0.65, n=3)
- Measurement outcomes
- 8,192 tokens · 4 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0
Source 5 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c5
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 5 streams: PP mean 891.46 t/s (min 890.02, max 894.18, SD 1.93, n=3); TG mean 130.54 t/s (min 130.36, max 130.65, SD 0.13, n=3); All mean 356.04 t/s (min 355.80, max 356.21, SD 0.17, n=3)
- Measurement outcomes
- 8,192 tokens · 5 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0
Source 6 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c6
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 6 streams: PP mean 883.00 t/s (min 880.03, max 885.00, SD 2.14, n=3); TG mean 134.39 t/s (min 134.32, max 134.48, SD 0.07, n=3); All mean 371.29 t/s (min 371.14, max 371.41, SD 0.11, n=3)
- Measurement outcomes
- 8,192 tokens · 6 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0
Source 7 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c7
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 7 streams: PP mean 895.78 t/s (min 870.78, max 934.21, SD 27.58, n=3); TG mean 134.38 t/s (min 133.22, max 135.14, SD 0.83, n=3); All mean 377.62 t/s (min 372.20, max 382.85, SD 4.35, n=3)
- Measurement outcomes
- 8,192 tokens · 7 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0
Source 8 · 2026-10-15
- Source run ID
- pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-off-c8
- Page entry ID
- gemma-4-12b-q8-0-single-gpu-20261016-off-b9022c91fa
- Entry sweep
- concurrency
- Raw entry label
- Gemma 4 12B Q8_0 — Single GPU — no splitting
- Configuration label
- Single GPU — no splitting
- Measured date
- 2026-10-15
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Requested devices
- CUDA0
- Split mode
- Single GPU — no splitting
- Placement detail
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 262,144 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- server durations aligned to client completion timestamps; aggregate-all uses client wall time
- Throughput spreads
- KV 8,192, 8 streams: PP mean 904.33 t/s (min 866.42, max 924.36, SD 26.82, n=3); TG mean 140.53 t/s (min 140.23, max 140.75, SD 0.22, n=3); All mean 398.80 t/s (min 393.29, max 401.83, SD 3.90, n=3)
- Measurement outcomes
- 8,192 tokens · 8 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0