research9 min read
Teaching a Small Transformer to Play Wordle
I trained a 7.2M Transformer on Wordle games. After SFT, DPO and GRPO, soft distillation and full-word scoring reached a 95.64% win rate on 719 known words.
Essays & Field Notes
Curiosity wherever it leads: technology, markets, AI, creativity, decision-making, and beyond.
I trained a 7.2M Transformer on Wordle games. After SFT, DPO and GRPO, soft distillation and full-word scoring reached a 95.64% win rate on 719 known words.
A technical and economic guide to HBM stack height, capacity, bandwidth, packaging yield, qualified high-speed supply, and the competing interpretations of lower-layer AI memory.
I weakened refusal in Qwen3.5-9B and found that most of the change came from one simple update in layer 15.
How I run FL Studio in a Windows 10 KVM on Ubuntu instead of dual-booting: the setup, the mistakes, the fixes, and the shortcuts that make the VM feel like a normal Linux app.