Gemma 4 E2B

google/gemma-4-E2B-it

2.3B active / 5.1B totalDenseText generationImage → textAudio → textText inputImage inputAudio input

Google · 4 recorded configurations

Measured setup

Hardware

  • 1× RTX 3090 · GPU-resident

Runtime and tool

  • live llama-server

Quantisation

  • UD-Q8_K_XL

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

1× RTX 3090 · llama.cpp

UD-Q8_K_XL · MTP 3

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Gemma 4 E2B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
08,639.81354.94
2,0487,786.02349.98
4,0967,401.53338.88
8,1926,727.11325.93
16,3845,625.34300.18
32,7684,297.29267.08
65,5362,922.45216.55
130,2401,774.83157.56

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Gemma 4 E2B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
28,1926,323.39478.75239.381,227.71
38,1925,831.64538.88179.631,432.69
48,1925,425.97587.84146.961,589.12
58,1925,336.50623.61124.721,692.45
68,1925,222.31638.58106.431,774.03
78,1925,128.62660.2194.321,846.09
88,1925,011.65670.8583.861,886.58
Technical record and provenance

Source 1 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-context-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c1
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
context
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
131,072 tokens
Actual context per slot
131,072 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 0, 1 stream: PP mean 8,639.81 t/s (min 8,625.05, max 8,647.63, SD 10.44, n=3); TG mean 354.94 t/s (min 354.21, max 356.33, SD 0.98, n=3); All mean 943.63 t/s (min 940.39, max 947.67, SD 3.03, n=3) · KV 2,048, 1 stream: PP mean 7,786.02 t/s (min 7,730.99, max 7,847.71, SD 47.88, n=3); TG mean 349.98 t/s (min 347.58, max 351.21, SD 1.70, n=3); All mean 923.50 t/s (min 918.81, max 926.26, SD 3.34, n=3) · KV 4,096, 1 stream: PP mean 7,401.53 t/s (min 7,359.81, max 7,482.65, SD 57.36, n=3); TG mean 338.88 t/s (min 338.08, max 339.57, SD 0.61, n=3); All mean 893.41 t/s (min 892.05, max 895.25, SD 1.35, n=3) · KV 8,192, 1 stream: PP mean 6,727.11 t/s (min 6,708.24, max 6,761.93, SD 24.65, n=3); TG mean 325.93 t/s (min 325.18, max 326.49, SD 0.56, n=3); All mean 854.12 t/s (min 851.87, max 855.47, SD 1.60, n=3) · KV 16,384, 1 stream: PP mean 5,625.34 t/s (min 5,608.62, max 5,645.29, SD 15.14, n=3); TG mean 300.18 t/s (min 297.13, max 302.33, SD 2.22, n=3); All mean 780.89 t/s (min 773.74, max 786.22, SD 5.26, n=3) · KV 32,768, 1 stream: PP mean 4,297.29 t/s (min 4,260.60, max 4,325.79, SD 27.24, n=3); TG mean 267.08 t/s (min 266.38, max 267.54, SD 0.50, n=3); All mean 684.42 t/s (min 683.61, max 684.96, SD 0.58, n=3) · KV 65,536, 1 stream: PP mean 2,922.45 t/s (min 2,916.96, max 2,930.96, SD 6.10, n=3); TG mean 216.55 t/s (min 216.04, max 217.09, SD 0.43, n=3); All mean 543.82 t/s (min 542.99, max 544.40, SD 0.60, n=3) · KV 130,240, 1 stream: PP mean 1,774.83 t/s (min 1,772.03, max 1,778.43, SD 2.67, n=3); TG mean 157.56 t/s (min 157.34, max 157.93, SD 0.27, n=3); All mean 386.08 t/s (min 385.40, max 386.63, SD 0.51, n=3)
Measurement outcomes
0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 130,240 tokens · 1 stream: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/e2b-context-gpu1

Source 2 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c2
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 2 streams: PP mean 6,323.39 t/s (min 6,108.24, max 6,470.94, SD 155.60, n=3); TG mean 478.75 t/s (min 468.72, max 484.58, SD 7.12, n=3); All mean 1,227.71 t/s (min 1,200.91, max 1,244.29, SD 19.13, n=3)
Measurement outcomes
8,192 tokens · 2 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 3 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c3
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 3 streams: PP mean 5,831.64 t/s (min 5,721.93, max 6,024.72, SD 136.95, n=3); TG mean 538.88 t/s (min 534.62, max 544.16, SD 3.96, n=3); All mean 1,432.69 t/s (min 1,427.22, max 1,437.42, SD 4.20, n=3)
Measurement outcomes
8,192 tokens · 3 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 4 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c4
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 4 streams: PP mean 5,425.97 t/s (min 5,411.33, max 5,439.17, SD 11.41, n=3); TG mean 587.84 t/s (min 587.52, max 588.18, SD 0.27, n=3); All mean 1,589.12 t/s (min 1,588.02, max 1,589.83, SD 0.79, n=3)
Measurement outcomes
8,192 tokens · 4 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 5 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c5
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 5 streams: PP mean 5,336.50 t/s (min 5,313.07, max 5,372.56, SD 25.88, n=3); TG mean 623.61 t/s (min 611.47, max 644.37, SD 14.75, n=3); All mean 1,692.45 t/s (min 1,688.53, max 1,699.29, SD 4.85, n=3)
Measurement outcomes
8,192 tokens · 5 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 6 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c6
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 6 streams: PP mean 5,222.31 t/s (min 5,200.98, max 5,259.05, SD 26.09, n=3); TG mean 638.58 t/s (min 637.08, max 639.51, SD 1.07, n=3); All mean 1,774.03 t/s (min 1,767.15, max 1,780.62, SD 5.50, n=3)
Measurement outcomes
8,192 tokens · 6 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 7 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c7
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 7 streams: PP mean 5,128.62 t/s (min 5,116.59, max 5,150.08, SD 15.22, n=3); TG mean 660.21 t/s (min 659.17, max 660.99, SD 0.76, n=3); All mean 1,846.09 t/s (min 1,842.84, max 1,848.81, SD 2.47, n=3)
Measurement outcomes
8,192 tokens · 7 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Source 8 · 2026-10-15

Source run ID
pub-20261016-trpro-gemma-e2b-q8-k-xl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-3-c8
Page entry ID
gemma-4-e2b-ud-q8-k-xl-single-gpu-20261016-mtp-3-5d78d5507b
Entry sweep
concurrency
Raw entry label
Gemma 4 E2B UD-Q8_K_XL — Single GPU — no splitting
Configuration label
Single GPU — no splitting
Measured date
2026-10-15
Hardware
1× NVIDIA GeForce RTX 3090
Runtime
llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement
gpu-resident
Observed devices
CUDA0
Requested devices
CUDA0
Split mode
Single GPU — no splitting
Placement detail
Single GPU — no splitting
KV-cache precision
Q8_0
Flash Attention
Enabled
Requested context per slot
9,024 tokens
Actual context per slot
9,216 tokens
Native model context
131,072 tokens
Batch size
512
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
MTP · draft max 3
Warm-ups
1
Measured repetitions
3
Timing method
server durations aligned to client completion timestamps; aggregate-all uses client wall time
Throughput spreads
KV 8,192, 8 streams: PP mean 5,011.65 t/s (min 5,005.09, max 5,023.38, SD 8.31, n=3); TG mean 670.85 t/s (min 666.53, max 673.32, SD 3.07, n=3); All mean 1,886.58 t/s (min 1,872.66, max 1,893.89, SD 9.85, n=3)
Measurement outcomes
8,192 tokens · 8 streams: measured
Notes
Automated llama-server KV-depth sweep (mtp-3, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU — no splitting.
Source article
/benchmarks/
Raw log path
research/benchmarks/logs/trpro/2026-10-15-independent-gpus/gemma-e2b-q8-k-xl-concurrency-physical1

Compare measured entries