Text so far
The best pet is a
Local AI
Local AI
Learn how models run, what hardware they need, and how to build a practical local inference system.
Start here
Three foundations for better decisions.
Choose hardware, understand model size, then connect the concepts into a complete inference loop.
01 / Buying guide
Best GPUs and Devices for Local AI Inference
Choose between GPUs, workstation cards, unified memory, and CPU inference using realistic constraints.
02 / FundamentalsLLM Quantization Without the Alphabet Soup
Understand model formats, bits per weight, memory tradeoffs, and why Q4 is a useful starting point.
03 / Learning pathHow Local LLMs Work: Models, Memory and Tokens
Follow the fundamentals path below from tokenization through memory, attention, and inference.
Interactive fundamentals
See the model’s work, step by step.
Use the explainers to inspect tokenization and cached attention, then estimate what a model will do on real hardware.
Interactive explainer
From prompt to next token
Step through the small loop a language model repeats while generating text.
Tokens and IDs
TheID 791bestID 1888petID 5207isID 374aID 264
The tokenizer splits text and gives every token a numeric ID.
Meaning and context
pet · 5207
[0.2, −0.7, …]Embeddings represent meaning. Attention mixes in useful context from the other tokens.
Possible next tokens
dog
cat
fish
One token is added
The best pet is a dog
Then the same loop starts again.
Step 1 of 5
Interactive explainer
Why generation uses a KV cache
Compare the work required to generate the same next token with and without saved attention data.
5 prompt tokens
Without a cache Old work was forgotten
Work this stepWaiting
Nothing is saved between steps.
With a KV cache Old attention data was saved
Saved K/V dataWork this stepWaiting
Earlier positions can be reused.
Interactive calculator
Open the calculator →Estimate model fit and generation speed.
Compare GPUs and unified-memory systems, check VRAM fit, and estimate tokens per second for your model.
Learn the fundamentals
Understand the inference loop.
Build a working mental model before comparing hardware or software.
- 01TokenizationInteractive explainer
- 02Parameters and model sizePlanned guide
- 03QuantizationFundamentals
- 04Context windowsPlanned guide
- 05KV cacheInteractive explainer
- 06Attention and mixture-of-expertsPlanned guide
Choose hardware
Match memory, bandwidth, and budget.
Start broad, then compare the devices and build types that fit your models.
Understand models
Read architecture claims in practical terms.
Connect model design to memory use, speed, quality, and local feasibility.
Build a local system
Turn compatible parts into a useful stack.
Start with the runtime choice, then wire up offloading, measurement, and remote access. Installation and security guides are next.
- 01Choosing a runtime: Ollama, llama.cpp or vLLMSoftware
- 02Linux installation guidesPlanned guide
- 03Model storage and downloadsPlanned guide
- 04GPU offloadingFundamentals
- 05Performance measurementCalculator
- 06Remote accessSoftware
- 07Security and privacyPlanned guide
Latest Local AI writing
Recent articles.
Chronological updates after the guided paths and interactive fundamentals.
Hardware10 min read
I Built a 48 GB VRAM Local AI Workstation with Two RTX 3090s
A dual RTX 3090 local AI workstation with 48 GB of distributed VRAM, Threadripper Pro, WRX80, 128 GB ECC RAM and enough expansion room for larger multi-GPU experiments.
Software8 min read
How to Access Your Local AI From Anywhere
A step-by-step guide to using Tailscale and SSH for private remote access to a home local AI server, with Ollama and Open WebUI examples.
Fundamentals8 min read
GPU Offloading for Local LLMs
A beginner's guide to splitting a local LLM between GPU VRAM and system RAM, with practical llama.cpp settings and advice for dense and mixture-of-experts models.
Software7 min read
Ollama, llama.cpp or vLLM: Which One Should You Use?
A beginner's guide to Ollama, llama.cpp and vLLM: what an inference engine does, how model formats fit in, and which tool suits a laptop, home server or busy API.
Fundamentals10 min read
Reasoning Models and Thinking Tokens Explained
A beginner's guide to chain-of-thought, reasoning models and test-time compute — from DeepSeek-R1 and GPT-o1 to the dot experiment and compressed reasoning — with practical advice for running distilled reasoning models on home hardware.
Fundamentals7 min read
Speculative Decoding and MTP Explained
A beginner's guide to speculative decoding, draft models and multi-token prediction, including when they speed up local LLMs and when the extra work slows them down.
Model analysis7 min read
Kimi K3 Is a 2.8-Trillion-Parameter Open-Weight Monster
Kimi K3 explained without the research-paper fog: 2.8T parameters, 104B active, Kimi Delta Attention, MXFP4 weights, benchmark results, API pricing, and what running it locally really takes.
Fundamentals8 min read
LLM Quantization Without the Alphabet Soup
A beginner's guide to local LLM quantization: what bits per weight means, how popular GGUF formats compare, when to choose Q2 through Q8, and why Q4 is a strong starting point.
Hardware10 min read
The Best Affordable Multi-GPU Motherboard for Local AI
A comparison of WRX80, TRX40, and AMD EPYC motherboards for multi-GPU local AI builds, with UK marketplace prices, PCIe lanes, memory bandwidth, Threadripper Pro CCD layouts, remote management, and the risks of importing used server hardware.
Model analysis10 min read
DeepSeek V4 Flash Is Almost Too Cheap to Meter
How DeepSeek V4 Flash cuts inference cost with sparse experts, compressed attention, FP4/FP8 weights, disk-backed prefix caching and speculative decoding, plus API cost estimates and real local benchmarks on RTX PRO 6000, DGX Spark and Apple Silicon.
Hardware10 min read
Apple Silicon Macs for Local LLMs
A concise guide to the M3 Ultra Mac Studio, M4 Max Mac Studio, and M5 Max MacBook Pro for local LLMs, including real benchmarks, current and used pricing, MLX and Metal software, Thunderbolt clustering, and an estimate for a possible M5 Ultra.
Hardware22 min read
NVIDIA DGX Spark and GB10 for Local LLMs
A practical guide to NVIDIA DGX Spark and other GB10 systems for local LLMs, covering real token speeds, memory bandwidth, MoE models, clustering, thermals, software support, pricing, Strix Halo, and Mac Studio alternatives.
Hardware10 min read
AMD Strix Halo for Local LLMs
A concise guide to AMD Strix Halo for local LLMs, covering real token speeds, memory bandwidth, MoE models, ROCm and Vulkan support, clustering, and the 192 GB Gorgon Halo refresh.
Buying guide22 min read
Best GPUs and Devices for Local AI Inference
A comparison of GPUs, workstation cards, unified-memory devices, and CPU inference setups for local AI, focused on price, speed, memory capacity, software support, and practical tradeoffs.
Benchmark20 min read
RTX 3090 for Local AI Inference
A practical look at the RTX 3090 for local AI workloads, covering specs, VRAM, bandwidth, TFLOPS, used pricing, power limiting, LLM performance, image generation, and why it remains one of the best-value GPUs for running open models at home.