GLM-4.5 Air 106B-A12B

zai-org/GLM-4.5-Air

12B active / 106B totalMoEText generationText input

Zhipu AI · 1 recorded configuration

Measured setup

Hardware

2× RTX 3090 · CPU offload

Runtime and tool

llama-bench

Quantisation

i1-Q4_K_S

MTP draft depth

MTP off

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

PP · left axisTG · right axis

View exact measurements

Exact context depth and measured fresh prefill, bulk prefix, and decode rates for GLM-4.5 Air 106B-A12B
Context depth (tokens)	Fresh prefill PP (tok/s)	TG (tok/s)
0	284.70	27.60
2,048	279.20	26.17
4,096	274.30	25.00
8,192	261.00	22.46
16,384	239.90	19.59
32,768	206.00	15.55

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for GLM-4.5 Air 106B-A12B
Streams	Aggregate PP (tok/s)	Aggregate TG (tok/s)	TG / stream (tok/s)	Aggregate all (tok/s)
1	284.70	27.60	—	—
1	289.60	27.14	27.14	68.57
2	289.90	18.56	9.28	49.34
4	295.80	27.51	6.88	69.59

Technical record and provenance

Run details

Source run ID: rtx3090x2-glm-4.5-air-106b-iq4ks-2026-08
Hardware: 2× RTX 3090
Runtime: llamaCpp · llama-bench fc6545d (build 50)
Weight placement: partially-offloaded
Split ratio: 65:35
CPU expert layers: 18
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 2,048
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: None (standard decoding)
Measured repetitions: 3
Notes: i1-Q4_K_S, batch 2048/512, 18 MoE expert layers on CPU, remaining layers split 65:35 (partially offloaded). Concurrency rows from llama-batched-bench (aggregate rates). Model is not in the calculator catalog, so this record is archival until it is added.
Source article: /local-ai/rtx-3090-for-local-ai-inference/

Compare measured entries