Gemma 4 31B

google/gemma-4-31B-it

30.7BDenseText generationImage → textText inputImage input

Google · 2 recorded configurations

Measured setup

Hardware

1× RTX 3090 · GPU-residentalso Runtime
2× RTX 3090 · GPU-resident

Runtime and tool

live llama-serveralso Hardware
llama-bench

Quantisation

Q4_K_M

MTP draft depth

MTP off

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

PP · left axisTG · right axis

View exact measurements

Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Gemma 4 31B QAT MTP
Context depth (tokens)	Fresh prefill PP (tok/s)	TG (tok/s)
0	1,517.60	50.80
2,048	1,425.30	49.21
4,096	1,392.30	48.60
8,192	1,325.30	47.38
16,384	1,213.20	45.25
32,768	1,025.70	41.49
65,536	788.60	35.56
131,072	541.20	27.66
262,144	332.50	19.08

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Gemma 4 31B QAT MTP
Streams	Aggregate PP (tok/s)	Aggregate TG (tok/s)	TG / stream (tok/s)	Aggregate all (tok/s)
1	1,517.60	50.80	—	—
1	1,442.10	50.10	50.10	140.53
2	1,446.60	85.73	42.87	229.94
4	1,473.10	114.99	28.75	298.39

Technical record and provenance

Run details

Source run ID: rtx3090x2-gemma-4-31b-q4km-2026-08
Hardware: 2× RTX 3090
Runtime: llamaCpp · llama-bench fc6545d (build 50)
Weight placement: gpu-resident
Split ratio: 1:1
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 2,048
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: None (standard decoding)
Measured repetitions: 3
Notes: Q4_K_M, batch 2048/512, all layers on GPU, tensor split 1:1. QAT MTP finetune by HauhauCS with speculative decoding disabled. Extended sweep to 64K/128K/256K. Concurrency rows from llama-batched-bench (aggregate rates).
Source article: /local-ai/rtx-3090-for-local-ai-inference/

Compare measured entries