Gemma 4 E2B

google/gemma-4-E2B-it

2.3B active / 5.1B totalDenseText generationImage → textAudio → textText inputImage inputAudio input

Google · 4 recorded configurations

Measured setup

Hardware

  • 1× RTX 3090 · GPU-resident

Runtime and tool

  • live llama-server

Quantisation

  • UD-Q8_K_XL

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

1× RTX 3090 · llama.cpp

UD-Q8_K_XL · MTP 1

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Gemma 4 E2B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
08,728.73243.68
2,0487,651.99231.02
4,0967,361.60232.17
8,1926,685.24219.66
16,3845,600.70206.06
32,7684,298.10187.32
65,5362,904.62155.97
130,2401,779.99116.11

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Gemma 4 E2B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
28,1926,195.60343.42171.71919.60
38,1925,569.91407.16135.721,115.23
48,1925,546.22448.73112.181,244.56
58,1925,386.44473.0494.611,325.29
68,1925,299.56501.0283.501,415.03
78,1925,154.86521.9174.561,482.32
88,1925,100.78536.9467.121,529.58
Technical record and provenance

Source 1 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-context-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c1
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
context
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
131,072 tokens
Actual context per slot
131,072 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 0, 1 stream: PP mean 8,728.73 t/s (min 8,654.64, max 8,771.18, SD 52.57, n=3); TG mean 243.68 t/s (min 243.38, max 244.06, SD 0.28, n=3); All mean 672.43 t/s (min 671.81, max 673.62, SD 0.84, n=3) · KV 2,048, 1 stream: PP mean 7,651.99 t/s (min 7,617.57, max 7,689.30, SD 29.36, n=3); TG mean 231.02 t/s (min 228.22, max 235.17, SD 2.99, n=3); All mean 636.14 t/s (min 629.24, max 646.55, SD 7.49, n=3) · KV 4,096, 1 stream: PP mean 7,361.60 t/s (min 7,317.42, max 7,401.84, SD 34.58, n=3); TG mean 232.17 t/s (min 231.58, max 232.71, SD 0.46, n=3); All mean 636.83 t/s (min 634.41, max 638.51, SD 1.75, n=3) · KV 8,192, 1 stream: PP mean 6,685.24 t/s (min 6,572.95, max 6,761.66, SD 81.11, n=3); TG mean 219.66 t/s (min 214.62, max 222.98, SD 3.62, n=3); All mean 601.33 t/s (min 588.27, max 609.79, SD 9.37, n=3) · KV 16,384, 1 stream: PP mean 5,600.70 t/s (min 5,566.06, max 5,666.10, SD 46.27, n=3); TG mean 206.06 t/s (min 201.33, max 210.46, SD 3.73, n=3); All mean 560.04 t/s (min 548.11, max 571.14, SD 9.42, n=3) · KV 32,768, 1 stream: PP mean 4,298.10 t/s (min 4,276.00, max 4,312.05, SD 15.80, n=3); TG mean 187.32 t/s (min 185.21, max 188.76, SD 1.52, n=3); All mean 502.52 t/s (min 497.01, max 506.50, SD 4.02, n=3) · KV 65,536, 1 stream: PP mean 2,904.62 t/s (min 2,851.88, max 2,933.93, SD 37.37, n=3); TG mean 155.97 t/s (min 155.83, max 156.26, SD 0.20, n=3); All mean 410.63 t/s (min 409.39, max 411.86, SD 1.01, n=3) · KV 130,240, 1 stream: PP mean 1,779.99 t/s (min 1,772.64, max 1,786.54, SD 5.70, n=3); TG mean 116.11 t/s (min 115.93, max 116.43, SD 0.23, n=3); All mean 299.36 t/s (min 298.87, max 300.26, SD 0.64, n=3)
Measurement outcomes
0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 130,240 tokens · 1 stream: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/e2b-context-gpu1

Source 2 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c2
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 2 streams: PP mean 6,195.60 t/s (min 6,177.86, max 6,212.46, SD 14.14, n=3); TG mean 343.42 t/s (min 341.32, max 344.56, SD 1.48, n=3); All mean 919.60 t/s (min 913.44, max 922.88, SD 4.36, n=3)
Measurement outcomes
8,192 tokens · 2 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 3 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c3
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 3 streams: PP mean 5,569.91 t/s (min 5,536.67, max 5,599.86, SD 25.90, n=3); TG mean 407.16 t/s (min 405.16, max 409.64, SD 1.86, n=3); All mean 1,115.23 t/s (min 1,109.35, max 1,121.99, SD 5.20, n=3)
Measurement outcomes
8,192 tokens · 3 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 4 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c4
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 4 streams: PP mean 5,546.22 t/s (min 5,370.91, max 5,655.71, SD 125.24, n=3); TG mean 448.73 t/s (min 448.35, max 449.00, SD 0.28, n=3); All mean 1,244.56 t/s (min 1,240.20, max 1,248.09, SD 3.27, n=3)
Measurement outcomes
8,192 tokens · 4 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 5 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c5
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 5 streams: PP mean 5,386.44 t/s (min 5,287.71, max 5,449.31, SD 70.68, n=3); TG mean 473.04 t/s (min 470.71, max 474.95, SD 1.76, n=3); All mean 1,325.29 t/s (min 1,323.26, max 1,328.52, SD 2.31, n=3)
Measurement outcomes
8,192 tokens · 5 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 6 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c6
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 6 streams: PP mean 5,299.56 t/s (min 5,270.77, max 5,320.27, SD 21.00, n=3); TG mean 501.02 t/s (min 498.05, max 503.42, SD 2.23, n=3); All mean 1,415.03 t/s (min 1,406.16, max 1,419.90, SD 6.28, n=3)
Measurement outcomes
8,192 tokens · 6 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 7 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c7
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 7 streams: PP mean 5,154.86 t/s (min 5,128.29, max 5,183.03, SD 22.38, n=3); TG mean 521.91 t/s (min 520.93, max 523.32, SD 1.02, n=3); All mean 1,482.32 t/s (min 1,479.40, max 1,485.15, SD 2.35, n=3)
Measurement outcomes
8,192 tokens · 7 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 8 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c8
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-1-3c6cd05243
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 1
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 8 streams: PP mean 5,100.78 t/s (min 5,084.74, max 5,120.19, SD 14.67, n=3); TG mean 536.94 t/s (min 528.64, max 541.71, SD 5.89, n=3); All mean 1,529.58 t/s (min 1,506.22, max 1,543.18, SD 16.59, n=3)
Measurement outcomes
8,192 tokens · 8 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-1, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Compare measured entries