Gemma 4 E2B
google/gemma-4-E2B-it
Google · 1 recorded configuration
Measured setup
Hardware
Runtime and tool
Quantisation
MTP draft depth
Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.
1× RTX 3090 · llama.cpp
UD-Q8_K_XL · MTP off
PP · left axisTG · right axis
View exact measurements
| Context depth (tokens) | Fresh prefill PP (tok/s) | TG (tok/s) |
|---|---|---|
| 0 | 9,390.74 | 161.48 |
| 2,048 | 8,331.66 | 158.07 |
| 4,096 | 7,988.08 | 154.75 |
| 8,192 | 7,191.03 | 149.03 |
| 16,384 | 6,125.78 | 141.46 |
| 32,768 | 4,694.88 | 127.29 |
| 65,536 | 3,218.15 | 107.36 |
| 131,072 | 1,960.39 | 80.58 |
Concurrent-stream measurements
| Streams | Context depth (tokens) | Aggregate PP (tok/s) | Aggregate TG (tok/s) | TG / stream (tok/s) | Aggregate all (tok/s) |
|---|---|---|---|---|---|
| 2 | 8,192 | 6,615.91 | 248.13 | 124.07 | 687.44 |
| 3 | 8,192 | 6,117.42 | 310.90 | 103.63 | 872.74 |
| 4 | 8,192 | 6,338.26 | 343.63 | 85.91 | 972.46 |
| 5 | 8,192 | 5,714.97 | 394.49 | 78.90 | 1,120.63 |
| 6 | 8,192 | 5,790.12 | 403.29 | 67.22 | 1,156.46 |
| 7 | 8,192 | 5,534.18 | 419.99 | 60.00 | 1,206.40 |
| 8 | 8,192 | 5,479.12 | 437.37 | 54.67 | 1,260.06 |
Technical record and provenance
Source 1 · 2026-10-10
This source allocated 132,096 tokens per slot, above its recorded native model context of 131,072 tokens. Allocation alone does not establish that the model can use context beyond its native limit.
- Source run ID
- trpro-gemma4-e2b-single-gpu-context-20261010-off
- Page entry ID
- trpro-gemma4-e2b-single-gpu-context-20261010-off
- Entry sweep
- context
- Raw entry label
- Single GPU · context and concurrency sweeps
- Configuration label
- Single GPU · fixed 132,096-token allocation · context extension
- Measured date
- 2026-10-10
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 476, commit 3466812) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Split mode
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 132,096 tokens
- Actual context per slot
- 132,096 tokens
- Native model context
- 131,072 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- Server PP/TG durations aligned to client completion timestamps; aggregate-all uses client wall time including SSH/HTTP.
- Throughput spreads
- KV 0, 1 stream: PP mean 9,390.74 t/s (min 9,315.36, max 9,488.16, SD 88.48, n=3); TG mean 161.48 t/s (min 161.13, max 161.89, SD 0.38, n=3); All mean 458.93 t/s (min 457.90, max 460.03, SD 1.07, n=3) · KV 2,048, 1 stream: PP mean 8,331.66 t/s (min 8,297.68, max 8,364.10, SD 33.24, n=3); TG mean 158.07 t/s (min 157.33, max 158.68, SD 0.69, n=3); All mean 447.89 t/s (min 445.81, max 449.28, SD 1.84, n=3) · KV 4,096, 1 stream: PP mean 7,988.08 t/s (min 7,948.09, max 8,031.88, SD 42.02, n=3); TG mean 154.75 t/s (min 154.61, max 154.98, SD 0.20, n=3); All mean 437.94 t/s (min 437.30, max 438.76, SD 0.75, n=3) · KV 8,192, 1 stream: PP mean 7,191.03 t/s (min 7,146.55, max 7,249.25, SD 52.71, n=3); TG mean 149.03 t/s (min 148.90, max 149.15, SD 0.12, n=3); All mean 420.70 t/s (min 420.11, max 421.13, SD 0.53, n=3) · KV 16,384, 1 stream: PP mean 6,125.78 t/s (min 6,104.32, max 6,156.73, SD 27.47, n=3); TG mean 141.46 t/s (min 141.12, max 141.70, SD 0.31, n=3); All mean 397.58 t/s (min 396.71, max 398.46, SD 0.87, n=3) · KV 32,768, 1 stream: PP mean 4,694.88 t/s (min 4,685.60, max 4,706.01, SD 10.33, n=3); TG mean 127.29 t/s (min 126.82, max 127.55, SD 0.41, n=3); All mean 354.75 t/s (min 353.39, max 355.49, SD 1.18, n=3) · KV 65,536, 1 stream: PP mean 3,218.15 t/s (min 3,211.74, max 3,221.50, SD 5.55, n=3); TG mean 107.36 t/s (min 106.97, max 107.68, SD 0.36, n=3); All mean 295.28 t/s (min 294.47, max 296.06, SD 0.80, n=3) · KV 131,072, 1 stream: PP mean 1,960.39 t/s (min 1,950.62, max 1,965.75, SD 8.48, n=3); TG mean 80.58 t/s (min 80.36, max 80.72, SD 0.19, n=3); All mean 218.68 t/s (min 218.04, max 219.10, SD 0.56, n=3)
- Measurement outcomes
- 0 tokens · 1 stream: measured; 2,048 tokens · 1 stream: measured; 4,096 tokens · 1 stream: measured; 8,192 tokens · 1 stream: measured; 16,384 tokens · 1 stream: measured; 32,768 tokens · 1 stream: measured; 65,536 tokens · 1 stream: measured; 131,072 tokens · 1 stream: measured
- Notes
- Automated llama-server KV-depth sweep (off). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Runtime GGUF metadata override: gemma4.context_length=int:132096. The downloaded model file is unchanged; throughput does not establish model quality beyond its native context. Single GPU CUDA0, split-mode none. The same attached process and 132,096-token allocation were retained across all eight depths. Native context is 131,072; runtime metadata override supplies PP/TG headroom, with unchanged RoPE. No quality claim beyond native context. Context and concurrency are presented in one entry, with their different allocation settings retained per source sweep.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-10-gemma4-e2b-single-gpu-context
Source 2 · 2026-10-10
- Source run ID
- trpro-gemma4-e2b-single-gpu-concurrency-20261010-off
- Page entry ID
- trpro-gemma4-e2b-single-gpu-context-20261010-off
- Entry sweep
- concurrency
- Raw entry label
- Single GPU · context and concurrency sweeps
- Configuration label
- Single GPU · 9,216 tokens per slot · concurrency
- Measured date
- 2026-10-10
- Hardware
- 1× NVIDIA GeForce RTX 3090
- Runtime
- llamaCpp · version: 0.3.0-dev (build 476, commit 3466812) built with GNU 15.2.0 for Linux x86_64
- Weight placement
- gpu-resident
- Observed devices
- CUDA0
- Split mode
- Single GPU — no splitting
- KV-cache precision
- Q8_0
- Flash Attention
- Enabled
- Requested context per slot
- 9,024 tokens
- Actual context per slot
- 9,216 tokens
- Native model context
- 131,072 tokens
- Batch size
- 512
- Microbatch size
- 512
- Prompt tokens
- 512
- Generated tokens
- 256
- Speculative decoding
- None (standard decoding)
- Warm-ups
- 1
- Measured repetitions
- 3
- Timing method
- Server PP/TG durations aligned to client completion timestamps; aggregate-all uses client wall time including SSH/HTTP.
- Throughput spreads
- KV 8,192, 2 streams: PP mean 6,615.91 t/s (min 6,383.91, max 6,742.02, SD 201.18, n=3); TG mean 248.13 t/s (min 246.58, max 249.48, SD 1.46, n=3); All mean 687.44 t/s (min 682.18, max 691.63, SD 4.82, n=3) · KV 8,192, 3 streams: PP mean 6,117.42 t/s (min 6,050.29, max 6,157.26, SD 58.47, n=3); TG mean 310.90 t/s (min 309.81, max 311.60, SD 0.95, n=3); All mean 872.74 t/s (min 869.47, max 874.48, SD 2.83, n=3) · KV 8,192, 4 streams: PP mean 6,338.26 t/s (min 5,945.90, max 6,534.75, SD 339.80, n=3); TG mean 343.63 t/s (min 342.29, max 344.51, SD 1.18, n=3); All mean 972.46 t/s (min 963.94, max 977.33, SD 7.41, n=3) · KV 8,192, 5 streams: PP mean 5,714.97 t/s (min 5,691.56, max 5,743.81, SD 26.54, n=3); TG mean 394.49 t/s (min 394.06, max 394.80, SD 0.38, n=3); All mean 1,120.63 t/s (min 1,118.96, max 1,121.94, SD 1.52, n=3) · KV 8,192, 6 streams: PP mean 5,790.12 t/s (min 5,563.01, max 5,910.41, SD 196.79, n=3); TG mean 403.29 t/s (min 400.44, max 406.91, SD 3.30, n=3); All mean 1,156.46 t/s (min 1,146.27, max 1,167.63, SD 10.72, n=3) · KV 8,192, 7 streams: PP mean 5,534.18 t/s (min 5,505.22, max 5,556.98, SD 26.42, n=3); TG mean 419.99 t/s (min 417.33, max 421.92, SD 2.38, n=3); All mean 1,206.40 t/s (min 1,199.59, max 1,211.85, SD 6.24, n=3) · KV 8,192, 8 streams: PP mean 5,479.12 t/s (min 5,454.76, max 5,503.68, SD 24.46, n=3); TG mean 437.37 t/s (min 434.81, max 439.53, SD 2.38, n=3); All mean 1,260.06 t/s (min 1,252.98, max 1,266.42, SD 6.75, n=3)
- Measurement outcomes
- 8,192 tokens · 2 streams: measured; 8,192 tokens · 3 streams: measured; 8,192 tokens · 4 streams: measured; 8,192 tokens · 5 streams: measured; 8,192 tokens · 6 streams: measured; 8,192 tokens · 7 streams: measured; 8,192 tokens · 8 streams: measured
- Notes
- Automated llama-server KV-depth sweep (off). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Single GPU CUDA0, split-mode none. Requested per-slot allocation 9,024 rounded by the engine to 9,216. Each stream-count configuration loads once. No single-stream row is copied from the larger context-sweep allocation.
- Source article
- /benchmarks/
- Raw log path
research/benchmarks/logs/trpro/2026-10-10-gemma4-e2b-single-gpu-concurrency