Gemma 4 12B

google/gemma-4-12B-it

11.95BDenseText generationImage → textAudio → textText inputImage inputAudio input

Google · 4 recorded configurations

Measured setup

Hardware

  • 1× RTX 3090 · GPU-resident

Runtime and tool

  • live llama-server

Quantisation

  • Q8_0

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

1× RTX 3090 · llama.cpp

Q8_0 · MTP 2

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Gemma 4 12B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
01,956.7639.71
2,0481,146.2738.12
4,0961,143.8951.27
8,1921,117.6449.93
16,3841,065.7065.28
32,768950.8665.58
65,536774.4155.61
131,072576.9325.91
261,312380.1519.33

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Gemma 4 12B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
28,1921,019.6881.0640.53213.11
38,192953.0893.8831.29251.80
48,192883.48100.7325.18275.49
58,192844.14111.6822.34306.57
68,192834.69123.2520.54340.39
78,192818.29117.2116.74328.34
88,192815.50132.3216.54370.71
Technical record and provenance

Source 1 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-context-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c1
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
context
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
262,144 tokens
Actual context per slot
262,144 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 0, 1 stream: PP mean 1,956.76 t/s (min 1,943.74, max 1,972.07, SD 11.68, n=3); TG mean 39.71 t/s (min 39.70, max 39.75, SD 0.02, n=3); All mean 113.91 t/s (min 113.84, max 113.98, SD 0.06, n=3) · KV 2,048, 1 stream: PP mean 1,146.27 t/s (min 1,142.01, max 1,151.11, SD 3.74, n=3); TG mean 38.12 t/s (min 38.11, max 38.14, SD 0.01, n=3); All mean 106.70 t/s (min 106.63, max 106.75, SD 0.05, n=3) · KV 4,096, 1 stream: PP mean 1,143.89 t/s (min 1,128.09, max 1,162.65, SD 14.26, n=3); TG mean 51.27 t/s (min 51.05, max 51.45, SD 0.17, n=3); All mean 140.23 t/s (min 139.66, max 140.85, SD 0.49, n=3) · KV 8,192, 1 stream: PP mean 1,117.64 t/s (min 1,096.70, max 1,133.09, SD 15.35, n=3); TG mean 49.93 t/s (min 48.15, max 50.89, SD 1.26, n=3); All mean 136.60 t/s (min 131.98, max 139.10, SD 3.27, n=3) · KV 16,384, 1 stream: PP mean 1,065.70 t/s (min 1,063.38, max 1,069.11, SD 2.46, n=3); TG mean 65.28 t/s (min 65.13, max 65.46, SD 0.14, n=3); All mean 172.85 t/s (min 172.51, max 173.34, SD 0.36, n=3) · KV 32,768, 1 stream: PP mean 950.86 t/s (min 948.73, max 952.21, SD 1.53, n=3); TG mean 65.58 t/s (min 65.45, max 65.68, SD 0.09, n=3); All mean 170.96 t/s (min 170.81, max 171.22, SD 0.19, n=3) · KV 65,536, 1 stream: PP mean 774.41 t/s (min 755.91, max 783.79, SD 13.09, n=3); TG mean 55.61 t/s (min 55.55, max 55.65, SD 0.04, n=3); All mean 144.24 t/s (min 143.59, max 144.64, SD 0.46, n=3) · KV 131,072, 1 stream: PP mean 576.93 t/s (min 573.26, max 579.24, SD 2.62, n=3); TG mean 25.91 t/s (min 25.71, max 26.06, SD 0.15, n=3); All mean 70.84 t/s (min 70.31, max 71.23, SD 0.39, n=3) · KV 261,312, 1 stream: PP mean 380.15 t/s (min 378.74, max 382.38, SD 1.59, n=3); TG mean 19.33 t/s (min 19.30, max 19.38, SD 0.04, n=3); All mean 52.26 t/s (min 52.15, max 52.37, SD 0.09, n=3)
Measurement outcomes
0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured; 261,312 tokens · 1 stream: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-gpu0-overlap-validation/gemma12-context-gpu0-validation

Source 2 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c2
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 2 streams: PP mean 1,019.68 t/s (min 952.32, max 1,085.05, SD 54.20, n=3); TG mean 81.06 t/s (min 71.42, max 86.23, SD 6.82, n=3); All mean 213.11 t/s (min 188.95, max 225.56, SD 17.09, n=3)
Measurement outcomes
8,192 tokens · 2 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 3 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c3
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 3 streams: PP mean 953.08 t/s (min 932.59, max 984.48, SD 22.55, n=3); TG mean 93.88 t/s (min 92.54, max 96.28, SD 1.70, n=3); All mean 251.80 t/s (min 246.32, max 261.86, SD 7.12, n=3)
Measurement outcomes
8,192 tokens · 3 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 4 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c4
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 4 streams: PP mean 883.48 t/s (min 848.62, max 952.64, SD 48.90, n=3); TG mean 100.73 t/s (min 94.86, max 104.81, SD 4.26, n=3); All mean 275.49 t/s (min 264.58, max 283.69, SD 8.03, n=3)
Measurement outcomes
8,192 tokens · 4 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 5 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c5
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 5 streams: PP mean 844.14 t/s (min 843.06, max 844.98, SD 0.81, n=3); TG mean 111.68 t/s (min 111.57, max 111.76, SD 0.08, n=3); All mean 306.57 t/s (min 306.26, max 306.78, SD 0.22, n=3)
Measurement outcomes
8,192 tokens · 5 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 6 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c6
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 6 streams: PP mean 834.69 t/s (min 830.63, max 839.58, SD 3.70, n=3); TG mean 123.25 t/s (min 123.01, max 123.52, SD 0.21, n=3); All mean 340.39 t/s (min 339.71, max 341.33, SD 0.69, n=3)
Measurement outcomes
8,192 tokens · 6 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 7 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c7
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 7 streams: PP mean 818.29 t/s (min 803.43, max 826.60, SD 10.53, n=3); TG mean 117.21 t/s (min 116.73, max 117.45, SD 0.34, n=3); All mean 328.34 t/s (min 326.68, max 329.22, SD 1.18, n=3)
Measurement outcomes
8,192 tokens · 7 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 8 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-2-c8
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-2-c56024faaf
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 2
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 8 streams: PP mean 815.50 t/s (min 807.76, max 819.39, SD 5.47, n=3); TG mean 132.32 t/s (min 132.06, max 132.47, SD 0.18, n=3); All mean 370.71 t/s (min 369.23, max 371.58, SD 1.05, n=3)
Measurement outcomes
8,192 tokens · 8 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-2, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Compare measured entries