Game development as a black box — the contract is the interface

Our designer typed one sentence: *the goalie should never dive outside his goal area, and he should never hold the ball in his hands outside it.* One working session later the keeper obeyed it in every situation we could think to throw at him, the sixty-odd behaviours we had already pinned down still held, and three design questions came back that only a person should answer. Nobody opened the keeper's code to read it. That is what this post is about.

It is a systems-theory story. Not the textbook diagrams — the one idea underneath them: you do not have to understand a system's insides to hold it accountable. You have to define its inputs, its outputs, and the contract between them.

The box moved

Before AI, input–output thinking stopped at the edge of the code. You could write down what a feature should do, but the only way to make the game actually do it was for someone to go inside and rework the internals by hand. The contract was a check you ran afterwards — if you ran it at all.

What changed is not that a model can write code. It is where the boundary sits. We designed the inside of our game once, so that every value in it can be read and every situation in it can be staged from the outside. With that in place, the whole development process becomes the system: one sentence of intent goes in, verified game behaviour comes out, and the designer works at the boundary instead of inside the box.

The contract is the interface

The glue between the designer and the box is a contract the game can fail. It names a situation, the variations that provoke it, and what must hold — exactly, every time, or as a rate over many runs, because a football match is never the same twice. It is not documentation. It runs, it goes red, and it stays after the work is done.

Most teams treat tests as the thing you write after the feature. Here the test *is* the feature request — written first, seen failing, and only then answered by a change to the game.

SENTENCE   "never dive outside his goal area"
           "never hold the ball in his hands outside it"

RESOLVE    which area?          -> the semicircle in front of his goal
           never, or rarely?    -> never: every run, every moment
           the cheap way out?   -> he could just stop diving.
                                   so: just inside the area,
                                   he must still dive.

CONTRACT   caught off his line, shots left / middle / right
           standing wide, a shot to the far side
           a loose ball he runs out for
           a catch he is carried out of the area with

EXPECT     zero moments diving outside
           zero moments with the ball in his hands outside
           inside: still dives, still catches, still wins the ball

Where the LLM sits

In a control loop the LLM is not the hero. It holds three jobs. As the *translator* it turns a vague sentence into a precise, checkable reference — the job that used to need an experienced engineer's judgement. As the *controller* it reads the error, decides where inside the game to act and how hard. As the *reporter* it hands back whatever the translation could not resolve.

The LLM is a black box too. It is non-deterministic, sometimes wrong, and — like any optimiser — it will take the cheap way out if the contract allows one. So we do not ask anyone to trust it. We ask it to satisfy contracts it cannot argue with.

What has to be true inside the box

Control theory has two old words for the preconditions of steering anything: *observable* — you can see the state — and *controllable* — you can put the system into the state you want. Most game code is neither, from the outside. A game is a pile of private state behind a render loop. We spent the year making ours both, on purpose, and that is the part we are proud of.

observable     the agent reads the live game's whole state while it runs
controllable   any situation staged by name, driven by real button presses
fast loop      a headless copy runs 1,600 situations in about ten seconds
native         the same code the player gets, minus the window
low noise      behaviour judged as a rate over many runs, not one take
stable         every earlier contract re-runs on every change
other sensors  its own invisible copy of the game, screenshots, QA runs
replayable     any moment of play saved and played back exactly
memory         every decision and its reasons, searchable by intent

None of these exist for the LLM's sake alone — a person debugging at midnight wants every one of them too. But together they turn a codebase into something a controller can close a loop around. Without them, the same model is a very fast typist guessing in the dark.

One sentence, end to end

Back to the keeper. The translation came first: the goal area became the semicircle in front of his goal, *never* became every run and every moment, and the contract grew controls nobody asked for — a keeper standing just inside his area must still dive, a keeper who runs out for a loose ball must still win it, a keeper on his line must still catch. Without those, a keeper who simply stopped diving would have passed.

Then the contracts went red, for the right reasons: caught off his line, he dived outside the area for well over a second; carried out of it with the ball, he held it in his hands for nearly four. Only then did anything inside the game change — where a dive may start and end, which balls count as catchable, the moment a held ball has to leave his hands.

Then the part that justifies the whole approach. Two older contracts — a near-post save, and a shot whose goal rate has to match its expected-goals value — went red. The new rule had quietly made the keeper concede shots he used to stop. Nobody remembered those contracts existed; they did not need to. The controller fixed its change, not the contracts, and all sixty-four went green again.

What came back to the designer was short and specific. A keeper caught off his line now concedes, because he may not dive there. He cannot block standing either, since blocking is an outfield skill. And one test had staged the keeper in a spot the new rule forbids — moved, with a note to check where the formation actually puts him. Decisions it was sure of, it made. Decisions it was not sure of, it left alone and listed. Every one of them was reported.

How the controller fails, and what catches it

We are not claiming a model that does not make mistakes. We are claiming a loop where mistakes are cheap to see.

• The cheap way out — a contract that a do-nothing change can satisfy will be satisfied that way. Controls inside the contract close the exit.
• Regressions — a change that fixes one behaviour and breaks another. Every earlier contract re-runs on every change, so the old behaviour defends itself.
• Silent design calls — a judgement the designer never saw. The report has a fixed place for every call made and every call left open.
• Broken house rules — in this very run it edited a file in a way our own rules forbid. A reviewer caught it, not a gate — which tells us where the next gate goes.
• What no contract sees — how a save looks, how the stick feels. Those still need eyes and hands; the lab only measures behaviour.

The metric is the transfer function

If the box is the process, the only question worth measuring is how faithfully input becomes output. Did the contract capture what the designer meant? How many questions came back, and were they real ones? How long from sentence to green, and how much of the designer's time sat inside that loop? Did anything that used to work stop working?

Those are the numbers we watch now. Not lines of code, not how clever the change was — the fidelity of one sentence turned into behaviour, and the residue that only a person can resolve. The insides still matter enormously. They just stopped being where the designer has to stand.

• Read next — The loop's ancestor — balancing a game while you play it. /devlog/balancing-while-you-play
• Why recording inputs, not pixels, lets a machine press play. /devlog/replay-a-machine-can-press-play
• Why a rule only holds when it can fail the build. /devlog/the-codebase-out-votes-the-prompt