League of Legends Season 1: First 5v5 Reinforcement Learning Match — Why Won't the AI Push the Nexus?

A Full 5v5 from Spawn: RL Agents Battle in the Real Client

Have you ever seen ten reinforcement learning agents play a complete 5v5 in League of Legends? Recently, a developer shared their experiment on Reddit: a single neural network controls all ten champions, battling from spawn inside the real game client, playing against itself. The camera follows the blue side's Ezreal.

The video shows the true state of the current policy: in one 15-minute match, no nexus or inhibitors were destroyed; in another 30-minute match, no nexus was taken, and the purple side's top inhibitor was destroyed four times (because it respawns). The result seems disappointing, but the author argues the interesting part is precisely the "why."

Experiment Environment: 2011 Client and Fan Reimplementation Server

This experiment runs on a 2011 client build (patch 1.0.0.126), with the server being a fan reimplementation called brokenwings — this is crucial because they have the server source code, allowing them to port the rules into a fast batch simulator and perform exact comparisons. The port took only 24 hours.

Validation results: 11 complete matches and 343 single-ability probes, with every unit field after every decision matching the server bit-for-bit — 354/354 all matched. The simulator runs at roughly hundreds of thousands of game ticks per second on a single 64-core GH200 (the engine itself can handle about 780k agent decisions per second); the C# server runs about 19k ticks per second across 64 games. The map fully includes lanes, minions, jungle monsters, towers, inhibitors, and nexuses, but currently no items, summoner spells, or runes.

In the video, the policy drives the real server in real time: each server frame → observation → ten actions → game commands, about 6,500 decisions per match.

"Fan reimplementation server" refers to a server-side program rewritten by community developers through reverse engineering of the official game protocol or based on leaked code. Such servers do not depend on official servers, can run independently, and allow developers to modify rules and record complete state data. brokenwings is one such project, recreating the logic of an early 2011 version of League of Legends. Having the server source code enables researchers to precisely port game rules into a high-speed simulator and compare simulator and real server states bit-for-bit, ensuring consistency between training and deployment environments.

Training: PPO+LSTM, One Network Controlling Ten Champions

Training uses PPO+LSTM, with one network controlling all ten champions (distinguished by one-hot encoding). Decisions are made every 133 milliseconds, fog of war is enabled, rewards are zero-sum, with shaping rewards for gold, experience, CS, health, structures, and kills.

Training scale: 4,096 parallel games, about 330k agent steps per second end-to-end, roughly 19 billion agent steps in about 30 hours on a single Lambda GH200.

The first task was teamfighting. But after about 100 rollouts, the policy converged to "don't fight": for both sides, stalling has zero value, and fighting is negative for the weaker composition, so "don't fight" is the Nash equilibrium under that reward — PPO found it. This was later resolved by adding full matches (with towers and CS). Most training matches are capped at 12 minutes of game time and start from "tower pressure" states (5–20 minutes in, champions leveled, outer towers damaged).

PPO (Proximal Policy Optimization) is one of the most commonly used policy gradient algorithms in reinforcement learning today. It avoids training collapse by limiting the magnitude of each policy update (using a clipped surrogate objective), achieving a good balance between sample efficiency and stability. LSTM (Long Short-Term Memory) is a recurrent neural network that can process sequential information and retain historical states. In this experiment, LSTM allows agents to remember past observations (e.g., where enemy champions disappeared, lane push status), enabling more reasonable decisions in the partially observable fog-of-war environment. Combining the two, PPO learns the policy while LSTM handles temporal dependencies — a common architectural choice in game AI.

Reward horizon refers to how far into the future an agent can "see" rewards when making decisions. With discount factor γ=0.995 and decisions every 133 milliseconds, the effective horizon is about 1/(1-γ) steps, i.e., about 200 steps, or roughly 27 seconds in time. This means the agent only acts for rewards obtainable within 27 seconds. If taking the nexus requires risking death by fountain laser and takes longer than 27 seconds, while killing a respawning inhibitor yields immediate reward within that horizon, then a myopic policy will prefer the latter. Adjusting γ or the decision interval can change the reward horizon, thereby guiding the agent toward longer-term goals.

What It Learned: Data and Limitations

Baseline test: with all ten champions idle, minion waves alone can take 6.83 towers, 0.01 inhibitors, and 0 nexuses per match. The trained policy adds about 0.5 towers, nearly every inhibitor, and every nexus kill seen in the video. Most tower kills in the video were actually done by minions.

Against the server's built-in intermediate bots: 55 wins, 8 draws, 1 loss in 64 matches. But note that about two bots per match often idle, so this is only a lower bound, not a strong opponent.

Against champions that do nothing, starting from spawn: within the 12-minute training cap, 0% of matches take the nexus; but within 30 minutes, 98% do, with a median of about 21 minutes. It can take the nexus — it was just never rewarded to do so.

The reason: γ=0.995 with decisions every 133 milliseconds means a reward horizon of about 27 seconds. Recapturing a respawning inhibitor pays off within that horizon every time; crossing the fountain laser to take the nexus does not. The ladder stalls here. The author recently restarted training with an 18-minute cap and a longer reward horizon; early rollouts show more nexus kills, but the experiment has only been running for a few hours.

Other Caveats and Interpretation

Side imbalance partly stems from the game content itself: in this build, purple side inner towers spawn with 1300 health while blue side has 2550 (a 0.01-second buff in the original script), and the server faithfully reproduces this. The team compositions are also not mirrored.

This is an early checkpoint, a single run on a single machine, with no human baseline yet.

My take: the value of this work isn't how well the AI plays, but that it validates the feasibility of bidirectional exact alignment between a real server and a fast simulator — 354/354 bit-for-bit consistency means almost no gap between training and deployment environments. At the same time, it clearly demonstrates how reward horizon shapes policy behavior: a 27-second myopia leads the AI to obsess over repeatable inhibitor kills rather than winning the game. This provides a clear direction for subsequent reward design adjustments.

According to the original Reddit post and the author's description, the entire project — from simulator porting to training to real-time deployment — demonstrates an engineering path for reinforcement learning in complex games. Although the current policy still has obvious limitations, the framework itself is worth watching.

One-line Summary

A single neural network controls ten champions in a real League of Legends client for a complete 5v5 match, revealing the profound impact of reward horizon on policy behavior.

Paragraph Summary

This article introduces a reinforcement learning experiment: a single PPO+LSTM network controls all ten champions, playing complete 5v5 matches from spawn on a 2011 League of Legends real client and the fan reimplementation server brokenwings. Researchers ported server rules into a fast simulator, achieving 354/354 bit-for-bit consistency, and completed about 19 billion steps of training on a single GH200. The policy achieved 55 wins, 8 draws, and 1 loss against built-in bots, but because γ=0.995 corresponds to only about a 27-second reward horizon, the AI prefers repeatedly killing respawning inhibitors over taking the nexus, resulting in 0% nexus takes within the training cap and 98% within 30 minutes. The experiment demonstrates the feasibility of exact alignment between real servers and simulators, and how reward design shapes policy behavior.