Wordle GPT

Problem

Teaching AI to play games often relies on external helpers, game engines, or huge models. Wordle GPT tests a simpler question: can a small language model learn the rules, track possible words, and make smart guesses using nothing but text predictions?

Approach

The model receives no access to a dictionary or game engine during play:

• Game text: Each game turns into a text stream of guesses and feedback: <G>could<F>22010<G>colon<F>22222<E>, where 0 is a miss, 1 is a misplaced letter, and 2 is an exact hit.
• Separating rules from strategy: Special markers tell the model whether to score a guess (learning the rules) or pick the next word (learning strategy).
• Preventing forgetting: Training only on expert guesses caused the model to forget the basic rules. Mixing in 5–10% rule-checking examples during training solved this without hurting guess quality.
• Valid word filtering: Restricting model guesses to real 5-letter words separated simple spelling mistakes from bad strategic choices.

Architecture

• Model: A small 4-layer transformer (3.2 million parameters).
• Training targets: Optimal next moves were calculated by a solver that picks guesses eliminating the most remaining candidate words.
• Dataset: Scaled from 100,000 to 1,000,000 unique board positions, mixing random play, simple guesses, and optimal moves.
• Evaluation: Tested across 719 hidden target words that were kept out of the training data.

Lessons Learned

• Rule learning is easy; strategy is hard: The model learned Wordle mechanics and feedback colors in just a few epochs, but choosing moves that efficiently narrow down hidden words took far more data.
• Spelling is rarely the issue: Limiting guesses to real dictionary words only recovered a few lost games; most losses came from unhelpful guesses rather than invalid words.
• Diminishing returns from data: Scaling from 200,000 to 1,000,000 games improved the win rate from 52.5% to 77.1%. However, each doubling yielded smaller gains, showing that the model’s small size (3.2M parameters) was becoming the main bottleneck.

View the project on GitHub. Read the full research breakdown in Teaching a Transformer to Play Wordle.