Record the inputs, not the pixels — a replay a machine can press play

A recorded session here is not a video. It is a short list of what you did — every tap, every drag, stamped with the exact instant it happened. Press play and the game does not load a film of the past. It *re-lives* it, move for move, from nothing but that list.

And the one pressing play does not have to be a person. That last part is the whole point — but to see why it matters, start somewhere mundane: the size of the file.

A recording that weighs almost nothing

A five-minute playthrough captured as video is a heavy, dim-witted thing: millions of pixels describing what a moment *looked like*, and nothing about what *caused* it. Capture the same five minutes as the inputs that drove it and you get a few kilobytes — a transcript of intentions. The pixels are gone, and it does not matter, because the game can draw them again.

This only works because the simulation is deterministic: feed it the same inputs, in the same order, on the same beats, and it produces the same run — every position, every number, exactly. The inputs are not a *hint* about what happened. They are the complete cause. Everything else is a consequence the machine can re-derive on demand.

What would that take in Unity, Unreal, or Godot?

The big engines all record *something*, and it is worth being precise about what. Unity's recorder captures video, image sequences, an animation clip — the output, for a human to watch back. Unreal ships a genuine replay system, but it records the networked game state, the stream a spectator or a kill-cam plays back, not your inputs. Godot leaves it to you. Three good answers to one question: *let a person watch it again.*

Reproducing a session *exactly*, from inputs alone, is a different question — and in a general-purpose engine it is a project, not a checkbox. You have to make the simulation deterministic first: a fixed timestep, disciplined randomness, the physics and the drawing pried apart so a slow frame never changes the outcome. Then capture at the input boundary. Then build the record-and-replay harness on top. Studios do it — competitive games with rollback netcode live or die on it — and it is routinely the hardest part of the build.

record a video        →  built in, minutes        →  watch it back
record network state   →  built in, moderate       →  spectate / kill-cam
replay from inputs     →  from scratch, the hard one →  regenerate the run

So the honest comparison is not *they cannot do this.* It is this: over there, deterministic input replay is a feature you build. Here, it is a property you already have.

It falls out of the architecture

I did not so much add replay to the engine as *notice* it was already there. The simulation was built input-driven and deterministic from the start — the same discipline a rollback-netcode game demands, where every machine must reach an identical state from identical inputs. Once the game already treats your inputs as the single source of truth, recording is just writing that truth down. Replaying is reading it back. The feature is a handful of small pieces because the expensive nine-tenths — the determinism — had already been paid for, as the price of admission for something else.

Under the hood, that single source of truth is literally shared memory. The main thread writes your taps and drags into one buffer — a SharedArrayBuffer, the web's shared-memory primitive. The render worker — which owns the camera, and so is the only realm that can turn a pixel into a target — writes what you actually clicked on into the same shared space. The simulation worker reads it as gospel. Three realms, one set of bytes, one boundary. Recording taps that boundary; replaying writes to it. There is no separate capture system drifting out of sync with the game, because the recorder and the simulation are reading the exact same source of truth.

The session becomes the bug report

*Works on my machine* is a sentence about missing information — the other person could not recreate your exact run. A deterministic input replay deletes the sentence. The bug report is no longer a video *of* the problem; it is the problem, replayable move-for-move on any machine. A regression test stops being a hand-written script and becomes *a real session you actually played*, re-run on every build to check the outcome has not drifted. And because the whole thing is a list of intentions, it is small enough to attach to a ticket.

And a machine can press play

Here is where determinism stops being a tidy engineering virtue and starts paying rent. If a session reproduces exactly from its inputs, then *anything* that can supply those inputs can drive the game — including an agent. It opens the game in a real browser, replays a recorded or scripted session to reach an exact situation, and then takes a performance capture, or inspects the resulting state, and reports back. No human re-enacting the run by hand.

*Reproduce it on my machine* stops being a request one engineer makes of another and becomes a command a machine runs against itself. The scenario is pinned, so a performance measurement is finally comparing like with like — the same work, every time, instead of a person trying to play *roughly the same way* twice and hoping. For an AI that is expected to test and profile its own changes, a replay it can press play on is not a convenience. It is the difference between guessing and knowing.

Conclusion

Recording intentions instead of pixels is a small feature with a large tell. It is what a codebase looks like when it is built to be driven by machines as much as by people — where the replay a developer records to understand a bug is, byte for byte, the replay an agent drives to test the fix. The engines that record the movie are answering *what did this look like.* I would rather be able to ask *what exactly happened* — and hand the question to something that can press play and go find out.