ERR_INTERNET_DISCONNECTED

No connection. So we taught the dino to run itself.

The same Deep Q-Network, pointed at a new environment. It gets one signal: stay alive, clear obstacles. It starts by running straight into the first cactus, then learns when to jump and when to duck. Press start and watch it learn.

The environment

speed 30 steps/f
Episode
0
Cleared · last
0
Best distance
0
Epsilon (explore)
1.00
Train steps
0
Replay buffer
0

What the agent values right now

run0.00
jump0.00
duck0.00

Q-values: predicted future reward for each action in the current state. The agent takes the highest (blue), except when it explores at random.

The learning curve

obstacles cleared per episode moving average (30)

It stays near zero while exploration is high, then lifts off as ε decays. Expect it to wobble even after it learns. That instability is real DQN behaviour, not a bug.

Same machinery as the Atari DQN

State7 numbers: distance to the next obstacle, its type and height, the dino's height and velocity.
Q-networkSmall MLP with hand-written forward pass, backprop, and Adam. No ML library.
Experience replayPast transitions stored and sampled in random batches to break correlation.
Target networkA frozen copy of the Q-network supplies the update target, synced periodically.
ε-greedyRandom early, then trust the learned policy as ε decays toward 0.02.
Bellman updateQ(s,a) ← r + γ·maxₐ′ Q_target(s′,a′), trained by gradient descent.

Difficulty levels · where it fails

With ramping on, the game speeds up the longer it survives and each band of speed is a level. Turn ramping off to train and compare at one fixed level, which gives a cleaner curve and a fair checkpoint comparison. The bars show, across recent runs, how far the dino gets before crashing. The red bar is the wall it keeps hitting. As it learns to beat a tier, watch that wall march to the right.

This runL1
Usually dies atL1
Furthest reachedL1

Checkpoints & frozen evaluation

No checkpoints saved yet.

Each bar is one frozen snapshot, evaluated with no learning and no exploration, averaged over many episodes. The thin line behind each bar marks its worst-to-best range. Watch for the tallest bar landing before the final checkpoint. That gap is the whole reason you keep snapshots.

Push the speed slider up, leave it a minute, and the dino starts clearing long runs. "Reset agent" wipes the brain so you can watch it learn from scratch.