Gemma 4 12B

google/gemma-4-12B-it

11.95BDenseText generationImage → textAudio → textText inputImage inputAudio input

Google · 4 recorded configurations

Measured setup

Hardware

  • 1× RTX 3090 · GPU-resident

Runtime and tool

  • live llama-server

Quantisation

  • Q8_0

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

1× RTX 3090 · llama.cpp

Q8_0 · MTP 3

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Gemma 4 12B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
01,933.9035.96
2,0481,184.2634.67
4,0961,155.8546.56
8,1921,105.3346.25
16,3841,067.2859.20
32,768931.7359.26
65,536770.2949.62
131,072574.5522.93
261,312375.9716.78

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Gemma 4 12B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
28,1921,006.7470.5335.26187.76
38,192928.1098.6432.88259.47
48,192921.98102.0125.50278.82
58,192874.3790.9118.18252.88
68,192879.33101.8916.98285.34
78,192865.81120.5917.23337.07
88,192858.0198.8212.35281.42
Technical record and provenance

Source 1 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-context-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c1
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
context
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
262,144 tokens
Actual context per slot
262,144 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 0, 1 stream: PP mean 1,933.90 t/s (min 1,917.35, max 1,947.50, SD 12.48, n=3); TG mean 35.96 t/s (min 35.94, max 35.98, SD 0.02, n=3); All mean 103.56 t/s (min 103.47, max 103.65, SD 0.07, n=3) · KV 2,048, 1 stream: PP mean 1,184.26 t/s (min 1,172.38, max 1,193.75, SD 8.89, n=3); TG mean 34.67 t/s (min 34.64, max 34.69, SD 0.02, n=3); All mean 97.85 t/s (min 97.73, max 97.92, SD 0.09, n=3) · KV 4,096, 1 stream: PP mean 1,155.85 t/s (min 1,115.98, max 1,177.45, SD 28.23, n=3); TG mean 46.56 t/s (min 46.48, max 46.66, SD 0.07, n=3); All mean 128.53 t/s (min 128.12, max 128.97, SD 0.35, n=3) · KV 8,192, 1 stream: PP mean 1,105.33 t/s (min 1,065.32, max 1,125.95, SD 28.29, n=3); TG mean 46.25 t/s (min 46.23, max 46.27, SD 0.01, n=3); All mean 127.29 t/s (min 126.91, max 127.52, SD 0.27, n=3) · KV 16,384, 1 stream: PP mean 1,067.28 t/s (min 1,065.22, max 1,069.24, SD 1.64, n=3); TG mean 59.20 t/s (min 59.08, max 59.36, SD 0.12, n=3); All mean 158.63 t/s (min 158.35, max 158.98, SD 0.26, n=3) · KV 32,768, 1 stream: PP mean 931.73 t/s (min 910.02, max 944.41, SD 15.42, n=3); TG mean 59.26 t/s (min 59.19, max 59.31, SD 0.05, n=3); All mean 156.33 t/s (min 155.78, max 156.61, SD 0.39, n=3) · KV 65,536, 1 stream: PP mean 770.29 t/s (min 752.20, max 784.20, SD 13.39, n=3); TG mean 49.62 t/s (min 49.60, max 49.64, SD 0.01, n=3); All mean 130.66 t/s (min 130.29, max 130.94, SD 0.28, n=3) · KV 131,072, 1 stream: PP mean 574.55 t/s (min 572.97, max 576.66, SD 1.55, n=3); TG mean 22.93 t/s (min 22.91, max 22.95, SD 0.02, n=3); All mean 63.32 t/s (min 63.28, max 63.37, SD 0.04, n=3) · KV 261,312, 1 stream: PP mean 375.97 t/s (min 371.02, max 379.06, SD 3.53, n=3); TG mean 16.78 t/s (min 16.73, max 16.82, SD 0.04, n=3); All mean 45.92 t/s (min 45.75, max 46.04, SD 0.12, n=3)
Measurement outcomes
0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured; 261,312 tokens · 1 stream: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-gpu0-overlap-validation/gemma12-context-gpu0-validation

Source 2 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c2
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 2 streams: PP mean 1,006.74 t/s (min 935.82, max 1,105.89, SD 72.24, n=3); TG mean 70.53 t/s (min 69.19, max 73.15, SD 1.86, n=3); All mean 187.76 t/s (min 183.06, max 196.28, SD 6.03, n=3)
Measurement outcomes
8,192 tokens · 2 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 3 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c3
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 3 streams: PP mean 928.10 t/s (min 911.35, max 944.86, SD 13.68, n=3); TG mean 98.64 t/s (min 98.47, max 98.89, SD 0.18, n=3); All mean 259.47 t/s (min 258.67, max 260.07, SD 0.59, n=3)
Measurement outcomes
8,192 tokens · 3 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 4 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c4
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 4 streams: PP mean 921.98 t/s (min 897.49, max 964.22, SD 29.99, n=3); TG mean 102.01 t/s (min 100.62, max 104.28, SD 1.62, n=3); All mean 278.82 t/s (min 273.52, max 288.52, SD 6.87, n=3)
Measurement outcomes
8,192 tokens · 4 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 5 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c5
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 5 streams: PP mean 874.37 t/s (min 859.35, max 888.86, SD 12.05, n=3); TG mean 90.91 t/s (min 90.52, max 91.14, SD 0.28, n=3); All mean 252.88 t/s (min 251.31, max 253.90, SD 1.12, n=3)
Measurement outcomes
8,192 tokens · 5 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 6 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c6
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 6 streams: PP mean 879.33 t/s (min 878.36, max 879.86, SD 0.69, n=3); TG mean 101.89 t/s (min 101.63, max 102.06, SD 0.19, n=3); All mean 285.34 t/s (min 284.70, max 285.81, SD 0.47, n=3)
Measurement outcomes
8,192 tokens · 6 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 7 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c7
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 7 streams: PP mean 865.81 t/s (min 859.35, max 873.65, SD 5.92, n=3); TG mean 120.59 t/s (min 120.46, max 120.84, SD 0.17, n=3); All mean 337.07 t/s (min 335.96, max 338.08, SD 0.87, n=3)
Measurement outcomes
8,192 tokens · 7 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 8 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-3-c8
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-3-7daeeaae74
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 8 streams: PP mean 858.01 t/s (min 835.76, max 871.73, SD 15.88, n=3); TG mean 98.82 t/s (min 98.09, max 99.25, SD 0.51, n=3); All mean 281.42 t/s (min 279.13, max 282.61, SD 1.62, n=3)
Measurement outcomes
8,192 tokens · 8 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Compare measured entries