Gemma 4 12B

google/gemma-4-12B-it

11.95BDenseText generationImage → textAudio → textText inputImage inputAudio input

Google · 4 recorded configurations

Measured setup

Hardware

  • 1× RTX 3090 · GPU-resident

Runtime and tool

  • live llama-server

Quantisation

  • Q8_0

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

1× RTX 3090 · llama.cpp

Q8_0 · MTP 1

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Gemma 4 12B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
01,948.4244.75
2,0481,186.0242.36
4,0961,172.0360.55
8,1921,128.2360.29
16,3841,067.7971.85
32,768931.8873.22
65,536780.4162.42
131,072567.6638.03
261,312380.3323.09

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Gemma 4 12B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
28,1921,059.7086.5943.29226.53
38,192912.1795.0331.68254.71
48,192887.18101.0125.25276.12
58,192908.94124.3024.86343.56
68,192853.60106.5817.76297.67
78,192846.77105.1415.02296.74
88,192841.98133.8016.73375.02
Technical record and provenance

Source 1 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-context-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c1
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
context
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
262,144 tokens
Actual context per slot
262,144 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 0, 1 stream: PP mean 1,948.42 t/s (min 1,935.34, max 1,969.06, SD 14.77, n=3); TG mean 44.75 t/s (min 44.51, max 45.00, SD 0.20, n=3); All mean 127.63 t/s (min 126.93, max 128.35, SD 0.58, n=3) · KV 2,048, 1 stream: PP mean 1,186.02 t/s (min 1,181.21, max 1,192.14, SD 4.56, n=3); TG mean 42.36 t/s (min 42.25, max 42.44, SD 0.08, n=3); All mean 117.97 t/s (min 117.67, max 118.19, SD 0.22, n=3) · KV 4,096, 1 stream: PP mean 1,172.03 t/s (min 1,167.06, max 1,176.77, SD 3.97, n=3); TG mean 60.55 t/s (min 60.19, max 60.73, SD 0.26, n=3); All mean 163.25 t/s (min 162.35, max 163.73, SD 0.64, n=3) · KV 8,192, 1 stream: PP mean 1,128.23 t/s (min 1,126.66, max 1,129.60, SD 1.21, n=3); TG mean 60.29 t/s (min 60.22, max 60.39, SD 0.07, n=3); All mean 162.13 t/s (min 161.98, max 162.36, SD 0.16, n=3) · KV 16,384, 1 stream: PP mean 1,067.79 t/s (min 1,063.54, max 1,072.64, SD 3.74, n=3); TG mean 71.85 t/s (min 71.74, max 71.90, SD 0.07, n=3); All mean 188.13 t/s (min 187.75, max 188.35, SD 0.28, n=3) · KV 32,768, 1 stream: PP mean 931.88 t/s (min 888.40, max 953.76, SD 30.75, n=3); TG mean 73.22 t/s (min 73.05, max 73.34, SD 0.12, n=3); All mean 187.74 t/s (min 186.73, max 188.54, SD 0.76, n=3) · KV 65,536, 1 stream: PP mean 780.41 t/s (min 773.98, max 784.64, SD 4.62, n=3); TG mean 62.42 t/s (min 62.34, max 62.47, SD 0.06, n=3); All mean 159.53 t/s (min 159.35, max 159.72, SD 0.15, n=3) · KV 131,072, 1 stream: PP mean 567.66 t/s (min 552.92, max 576.36, SD 10.47, n=3); TG mean 38.03 t/s (min 37.99, max 38.08, SD 0.04, n=3); All mean 99.59 t/s (min 99.44, max 99.68, SD 0.11, n=3) · KV 261,312, 1 stream: PP mean 380.33 t/s (min 377.46, max 382.40, SD 2.10, n=3); TG mean 23.09 t/s (min 22.97, max 23.17, SD 0.09, n=3); All mean 61.19 t/s (min 60.85, max 61.43, SD 0.25, n=3)
Measurement outcomes
0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured; 261,312 tokens · 1 stream: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-gpu0-overlap-validation/gemma12-context-gpu0-validation

Source 2 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c2
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 2 streams: PP mean 1,059.70 t/s (min 970.83, max 1,108.54, SD 62.94, n=3); TG mean 86.59 t/s (min 80.07, max 91.52, SD 4.81, n=3); All mean 226.53 t/s (min 212.78, max 234.79, SD 9.79, n=3)
Measurement outcomes
8,192 tokens · 2 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 3 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c3
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 3 streams: PP mean 912.17 t/s (min 838.59, max 994.59, SD 63.99, n=3); TG mean 95.03 t/s (min 82.87, max 101.90, SD 8.62, n=3); All mean 254.71 t/s (min 221.99, max 275.63, SD 23.44, n=3)
Measurement outcomes
8,192 tokens · 3 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 4 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c4
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 4 streams: PP mean 887.18 t/s (min 845.22, max 954.88, SD 48.33, n=3); TG mean 101.01 t/s (min 98.08, max 105.22, SD 3.05, n=3); All mean 276.12 t/s (min 266.60, max 290.85, SD 10.56, n=3)
Measurement outcomes
8,192 tokens · 4 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 5 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c5
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 5 streams: PP mean 908.94 t/s (min 892.33, max 926.30, SD 13.88, n=3); TG mean 124.30 t/s (min 123.00, max 126.42, SD 1.52, n=3); All mean 343.56 t/s (min 337.43, max 350.69, SD 5.46, n=3)
Measurement outcomes
8,192 tokens · 5 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 6 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c6
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 6 streams: PP mean 853.60 t/s (min 850.84, max 857.80, SD 3.02, n=3); TG mean 106.58 t/s (min 105.81, max 107.15, SD 0.56, n=3); All mean 297.67 t/s (min 295.55, max 298.99, SD 1.52, n=3)
Measurement outcomes
8,192 tokens · 6 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 7 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c7
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 7 streams: PP mean 846.77 t/s (min 845.38, max 847.92, SD 1.05, n=3); TG mean 105.14 t/s (min 104.61, max 105.46, SD 0.38, n=3); All mean 296.74 t/s (min 295.37, max 297.44, SD 0.97, n=3)
Measurement outcomes
8,192 tokens · 7 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Source 8 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma12-q8-0-concurrency-2026-10-14-single-gpu-isolated-co-schedule-validation-mtp-1-c8
Page entry ID
gemma-4-12b-q8-0-single-gpu-20261016-mtp-1-26e3237b99
Entry sweep
concurrency
Raw entry label
Gemma 4 12B Q8_0 — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
262,144 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 8 streams: PP mean 841.98 t/s (min 840.48, max 844.00, SD 1.48, n=3); TG mean 133.80 t/s (min 133.77, max 133.88, SD 0.06, n=3); All mean 375.02 t/s (min 374.79, max 375.33, SD 0.23, n=3)
Measurement outcomes
8,192 tokens · 8 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/gemma12-q8-0-concurrency-physical0

Compare measured entries