System One / System Two

System One / System Two

A tiny network, trained by a slow planner, proposes every move with a confidence. A cheap check catches its dangerous moves, and the planner decides whenever the network is unsure.

How it works

System One, System Two, the guard, and what the percentages mean

System Two is a planner. Before every move it looks ahead, by searching or simulating, from each candidate action and picks the best. It is strong but slow; its work is counted in compute units: the positions it explores or the physics steps it simulates.

System One is a small neural network. From the game's state (its inputs are listed under “This game”), it proposes a move in a single pass, 1 unit (5 for an ensemble of five networks), together with a confidence.

The threshold and the guard. If the confidence is below the threshold (the slider), the planner decides instead. If it is above, a cheap guard checks the move for immediate danger; if the move fails, the planner decides too.

How System One learns. It first imitates the planner on games the planner plays. Then the hybrid plays, and every state where it had to ask the planner is labelled by the planner and added to its training data. The network gets more data where it was unsure or wrong.

The percentages. “Who decided” counts the last 1,000 moves. With untrained (random) weights the confidence is low, so the planner decides almost everything: System Two near 100%, System One near 0%. As the network trains, its confidence grows and System One's share rises (watch it with “Train in this tab”, and in the “Share of moves” chart). How far it rises depends on the game: for the lander and racing, the planner still decides most moves at the end of training. Moving the threshold down gives more moves to System One: cheaper, but riskier.

The cost. The large number is the mean compute per move over the same 1,000 moves. The “Cost vs quality” chart compares it with the planner alone, from the published study.

score: 0
episode: 1
avg last 10: –
moves/s: –

Who decided

–

System Onenetwork alone · 1 unit –
Guard stopped itcheap check → planner –
System Twolow confidence → planner –

– compute units per move, now

Over the last 1,000 moves. The cost of a search depends on the state (a crowded Snake board is cheap to search), so compare with the averages only over whole games.

System One's policy

confidence threshold: below it, the planner decides

Controls

Model: untrained (random weights)

Confidence threshold 0.70 Speed 60 moves/s

Guard Seed

This game

What happens

The network

Outputs

Inputs

The planner (System Two)

The guard

Published result

Inside System One

The live forward pass behind the current decision. Nodes light up with their real activations, and only the connections that contribute most to this decision are drawn (positive, negative); into the outputs, those into the chosen action and the runner-up. Drag to orbit. The view re-runs the forward pass for display; that work is not counted in the cost.

Training in this tab

Idle. Training takes one to two minutes on a laptop and runs in a background worker.

Share of moves per iteration

escalated by the guard agreement with planner

Mean episode score per iteration

Cost vs quality

Published results.

planner alone hybrid, no guard hybrid + guard other / baseline live game