Llama 3.3 70B

meta-llama/Llama-3.3-70B-Instruct

70BDenseText generationText input

Meta · 1 recorded configuration

Measured setup

Hardware

  • 2× RTX 3090 · GPU-resident

Runtime and tool

  • llama-bench

Quantisation

  • Q4_K_S

MTP draft depth

  • MTP off

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

2× RTX 3090 · llama.cpp

Q4_K_S · MTP off

PP · left axisTG · right axis
View exact measurements
Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Llama 3.3 70B
Context depth (tokens)Fresh prefill PP (tok/s)TG (tok/s)
0786.6031.91
2,048746.5030.27
4,096721.2028.95
8,192673.7026.73
16,384598.4023.30
32,768486.0018.50

Concurrent-stream measurements

Concurrent-stream throughput at the recorded existing-prefix depth for Llama 3.3 70B
StreamsContext depth (tokens)Aggregate PP (tok/s)Aggregate TG (tok/s)TG / stream (tok/s)Aggregate all (tok/s)
10786.6031.91——
10750.4031.0131.0185.94
20752.5049.8424.92132.03
40751.8062.9315.73161.71
Technical record and provenance

Run details

Source run ID
rtx3090x2-llama-3.3-70b-q4ks-2026-08
Hardware
2× RTX 3090
Runtime
llamaCpp · llama-bench fc6545d (build 50)
Weight placement
gpu-resident
Split ratio
1:1
KV-cache precision
Q8_0
Flash Attention
Enabled
Batch size
1,024
Microbatch size
512
Prompt tokens
512
Generated tokens
256
Speculative decoding
None (standard decoding)
Measured repetitions
3
Notes
Q4_K_S, batch 1024/512, all layers on GPU, tensor split 1:1. Concurrency rows from llama-batched-bench (aggregate rates).
Source article
/local-ai/rtx-3090-for-local-ai-inference/

Compare measured entries