Video models become the second base for robot policies

A year ago the daily Scholar Inbox digest of robotics papers was almost entirely VLA — vision-language-action models built on a pretrained vision-language backbone. Now a second acronym turns up nearly every day: WAM, world-action models, which start from a pretrained video or world model instead and predict future states and robot actions together. In October 2025, in a piece called "The State of VLA", the same author who is now mapping the field called WAM a small corner inside VLA research, well behind VLM-based policies in popularity. Less than a year later, that author expects WAM to become the second major recipe for robot foundation models.

The two building blocks are easy to state on their own. A visuomotor policy takes current observations plus a goal or instruction and emits robot actions. A world model takes a current state and an action — or an abstract description of one — and predicts the future visual or latent state. A WAM sits at the intersection: the pretrained video or world model supplies the initial representation, and the system forecasts future states and robot actions at the same time.

There are two explanations for the surge, and the author offers both. The cheap one is that WAM is the open lane — a new direction many teams want a paper in, at a moment when the VLA naming space is running out, since X-VLA and Ego-VLA are already taken. The structural one, which the author considers more likely, is that VLM-based VLAs have hit their limits.

That second explanation has receipts. Modern VLAs inherit the benefits of internet-scale image and text pretraining and still struggle at exactly the point where language has to become movement. The link between words, pixels and behaviour has to be learned from robot data, the scarcest supply in the stack. Several papers report that the underlying VLM's capabilities degrade during that transfer, or were explicitly engineered around so they would not — particularly when the action-learning objective diverges sharply from what the VLM was trained for. VLM2VLA names the failure outright as catastrophic forgetting in the move from VLM to VLA. Knowledge Insulation traces it to architecture and detaches the gradients of the flow-matching action module from the VLM backbone to protect what the model already knew about language and images; the authors report better training convergence, better task scores and better instruction following. Co-training and discrete action tokenizers have softened the problem. They have not removed it.

So the field now runs two competing ways of putting knowledge into a robot foundation model. The first is the established VLA recipe, shaped by Pi-0 and refined in Pi-0.5, in which policy learning starts from a VLM. It is what NVIDIA's open GR00T work, Xiaomi Robotics, Being-H0.5 and others build on.

The second arrived later: use pretrained video models as an alternative route to general-purpose manipulation. The public examples include DreamZero and Cosmos Policy from NVIDIA, LingBot-VA from Ant Group, DVA from Rhoda AI, Cortex 2.0 from Sereact and mimic-video from Mimic Robotics. University labs and open research groups are pushing the same idea through Video Prediction Policy, Unified Video Action Model and Fast-WAM.

Worth reading that list twice: NVIDIA appears on both sides of the split, with GR00T on the VLA side and DreamZero and Cosmos Policy on the video side. That matters because the author's own argument is that nobody gets to hedge. The choice of base propagates through the training recipe, the data mix and inference optimisation, and running these models at scale is expensive enough that most teams will have to commit to one path rather than develop both properly in parallel. Whether the two converge is unknown. Hedging, on this account, is a privilege of whoever can afford two full training stacks.

The argument for starting from video is that the representation already contains the hard part. Video models are trained on large corpora and learn spatio-temporal structure — how visual scenes change — and most are text-conditioned, generating from a detailed description, sometimes with a starting frame and sometimes without. Much of that footage shows intentional behaviour: hands reaching for objects, tools moving, objects interacting, scenes changing because a human or a robot acted. Before the model has seen a single robot action label, it has encoded a mapping between language, visual change and plausible object interaction.

How much of that is already there? To probe it, the authors ran one unoptimised attempt with Google's Veo 3.1, currently one of the strongest video generation models. They took a single context frame from a RoboArena run of the toaster task in the DROID setup, then asked Veo to first press the toaster lever — matching the original task and the DROID demonstration — and afterwards pick up the orange to the left of the toaster, a composed continuation that appears in no demonstration. The odds that this specific run sat in Veo's training data are very low, though the training set cannot be inspected, so this is a qualitative check on the base representation rather than a controlled membership study.

With no robotics fine-tuning at all, Veo 3.1 generated the run zero-shot, and for a model never trained to be a robot policy the result is conspicuously plausible: smooth motion, a stable background, the arm travelling to both targets along a sensible trajectory, and the instructed order respected — lever first, orange second.

The failures are just as visible. The lever is never fully depressed, and in places the motion suggests the model is trying to pull it upward. More telling, the two-finger gripper from the DROID rig becomes a four-fingered hand, and almost immediately after the first frame the fixed-base arm is reimagined as a different robot with fewer degrees of freedom.

Those artifacts are the whole argument in one frame. The model has learned what robot manipulation looks like; it has not learned what this robot is. It is leaning on general visual priors rather than modelling specific hardware, which is why the clip reads as convincing and would be useless as a control signal. Fine-tuning is supposed to convert that imagination into reliable control — and this is where the field's evidence thins out fastest. The author is careful to label the core claims as hypotheses rather than settled results, supported by qualitative observation, simulation numbers and a few early real-world signals, with no clean head-to-head comparisons yet. A research community is reorganising around a new base before anyone has published a controlled comparison against the base it is leaving.

The more interesting question is why now. UniPi proposed essentially this recipe in 2023, and the author asks directly why several years passed before it went mainstream. The answer visible in the Veo demonstration is that the idea was never the bottleneck — the video models were. WAM became viable when generation quality crossed the line where a rollout is worth conditioning on. That makes robot foundation models a downstream consumer of the consumer video generation cycle, which is a strange dependency for a field whose hard constraint is supposed to be real-world data.

Unlike VLA, where the recipe has largely converged on co-training a VLM with a flow transformer for action generation, WAM is still expanding in several directions at once. Nobody knows which combination of architectural choices wins, or whether the successful systems will mix elements from several.

The author organises the space along three axes, not fully independent of one another, and notes that several models fit no single category — this is a practical map for reading current papers, not a taxonomy. The first axis is the policy formulation: what the model predicts, and how predicted video becomes actions. Three variants differ at the inference boundary — inverse dynamics, joint prediction, and representation-only.

Inverse dynamics is the clearest version: imagine the future, then infer from that video which action most likely produced it. It moves the hard language-to-action binding into the video generation step, where a command has to become a plausible visual change, on the bet that video pretraining already covers the language-to-visual-change half. The action module then only has to solve inverse dynamics rather than relearn everything from robot demonstrations.

The author's bet is that WAM becomes the second standing recipe alongside VLM-based VLA, and that a hybrid ultimately wins. That is probably right and also the cheapest available prediction — hybrids usually win. The expensive part is the interval before it happens, when every serious lab has to pick a base, fund one training stack and discover two years later whether it backed the representation that transfers. The Veo clip shows how much a video model already knows for free. It also shows that what it knows is a hand with the wrong number of fingers.