The Simulator a Swarm Dreams With

Watch a person reach for a cup near the edge of a table and you will see them slow down a fraction of a second before contact. They are not reacting to anything yet. They are running a small forecast: if my hand keeps moving at this speed, the cup tips. That forecast, performed silently and constantly, is a world model. It is the thing a robot has almost always lacked, and the thing the last two years of research have been quietly building.

Policy versus model

Most robots, and most of what gets called AI in robotics, are policies. A policy is a function from observation to action: it sees the current frame and outputs the next motor command. It is fast and it is blind. It does not know what its own action will cause, only that this action tended to work in situations that looked like this one. A world model is the opposite kind of object. It is a function from a state and an action to the next state. It does not tell the robot what to do. It tells the robot what would happen if it did a given thing. With one of those, a robot can try a thousand actions in imagination, score the futures they lead to, and only then move.

A policy reacts to the present. A world model lets the robot visit the future, cheaply, before committing its body to it.

Predicting in a latent space, not in pixels

The naive way to imagine the next second is to predict the next video frame, pixel by pixel. This is ruinous. Most of a frame is detail that does not matter for acting, the exact grain of the table, the flicker of a shadow, and a model that spends its capacity rendering those is slow and brittle. The better idea, and the one that defines the current generation, is to predict in a compact latent space. The robot compresses what it sees into a short vector of meaning, an embedding, and the world model predicts the next embedding rather than the next image. Meta's V-JEPA 2, released in June 2025, is built exactly this way: an encoder turns video into embeddings, and a predictor learns to say what the next embedding will be. It learns the gist of what happens next, not the picture of it.

This is the JEPA idea, joint-embedding predictive architecture. You never ask the model to reconstruct the world. You ask it to predict its own representation of the world, one step ahead, and you train it by penalizing how wrong that prediction was in embedding space. The payoff is that imagining the future becomes a few matrix multiplications on a short vector instead of the painting of a frame, which is what makes it fast enough to do hundreds of times before each real move.

Where the actions come from

A world model is only useful for planning if you can condition it on actions, if you can ask not just what happens next but what happens next if I push left. The trouble is that the internet is full of video and almost none of it is labeled with the actions that produced it. The resolution, worked out in methods like Latent Action Pretraining, is to learn the actions too. A small model looks at consecutive frames and invents a discrete latent action that explains the change between them, a guessed verb for an unlabeled motion. The world model can then be trained on vast amounts of unlabeled human video, and only at the end is that invented vocabulary mapped onto a particular robot's real motors using a few hours of teleoperation. V-JEPA 2 reached usable pick-and-place from sixty-two hours of robot data on top of a million hours of passive watching.

The most vivid version of this is Genie. DeepMind's Genie 3, shown in August 2025, generates an entire interactive world from a text prompt or an image, a place you can walk through at twenty-four frames a second, where what you saw a moment ago is still there when you turn back, held consistent for minutes. It learned to be controllable from video alone, never told which button did what, and inferred the latent actions that make a world respond to input. It is a world model you can step inside, which is precisely the kind of place you can let a robot practice.

Why a swarm dreams together

Now give the same world model to many robots. Each one can imagine the next few seconds, but more importantly they can imagine the same next few seconds, because they are running the same learned physics on a shared picture of the scene. This is what lets a swarm coordinate without a controller. If every drone can predict where the others will be in two seconds, given the maneuver each has announced, then they do not need to negotiate every move out loud. They agree on a single predicted future and act against it, the way a flock turns as one. Coordination becomes a matter of consensus on one imagined world rather than a storm of messages, and recovery becomes local: when reality diverges from the dream, each agent re-imagines and adjusts, no human in the loop.

Many agents agree on one future not by talking more, but by sharing the same imagination and running it forward in lockstep.

The dream drifts

All of this has a hard ceiling, and it is honest to name it. Every imagined step is built on the last one, so every small error compounds. Predict one second ahead and the dream is sharp; predict ten and the model is now imagining on top of its own mistakes, and the trajectory peels away from anything the real world would do. A contact misjudged by a millimeter, a friction guessed slightly wrong, and the plan that looked flawless in imagination fails on contact with the floor. This is the sim-to-real gap restated from the inside: a model can dream a future that is internally coherent and physically false. The discipline that keeps these systems usable is to imagine only a short horizon, act, look at what actually happened, and dream again, never trusting the forecast past the few seconds where it is still tethered to the world.

So the promise and the limit are the same fact. A robot that can imagine can plan, coordinate, and recover on its own. But it can only imagine accurately for a moment, and the whole art is in acting inside that moment, then re-grounding before the dream drifts too far. A swarm is coordinated for exactly as long as it shares a future that is still close enough to true.