Runway opens access to its Praxis-1 robot control model

How Praxis-1 controls robots

Runway first introduced Praxis-1 in late September, but access was limited to selected partners. The company has now published the model’s weights, source code and a detailed research report with new test results.

Praxis-1 takes camera images, text instructions and information about the robot’s position. It uses these inputs to predict how the surrounding environment will change and calculate suitable movements. As new frames arrive, the model updates its control.

Praxis-1 is built on a video model trained to understand motion and object interactions from ordinary video footage. Runway then fine-tuned it on videos of human actions and robot data. This approach could reduce the need for expensive datasets collected specifically for robotics training, although Praxis-1 still requires additional training on data from the specific robots it will control.

Runway is known for its generative video models, including Gen-4.5. The company describes Praxis-1 as a world action model: it predicts changes in a scene and uses them to determine a robot’s movements. Since September, robotics companies Standard Bots, Noble Machines and Ultra have tested the model on a humanoid robot, an industrial robotic arm and a two-armed warehouse robot.

Test results

To adapt Praxis-1 to a Franka robotic arm, Runway used about 277 hours of recordings from the publicly available DROID research dataset. The recordings include robots grasping and moving objects, for example. For a two-armed robot, the company used between 1,400 and 2,600 hours of training material. DROID data and data from other robotic systems were also used in earlier stages of training. It remains unclear whether Praxis-1 requires less robot data overall than comparable models that have not been pretrained on video.

In Nvidia’s RoboLab-120 ranking, the model achieved a 46.2% task success rate. The benchmark tests how reliably models complete 120 tasks in simulation, including sorting, grasping and stacking objects. FLUX 3 Action, also built as a world action model, scored 42.9% under the same conditions. Praxis-1 ranked first among models in this category and fifth overall, behind two AI agents and two robotics models with different architectures. Runway conducted all the tests; the results have not yet been independently verified.

In a separate test on a real Franka robotic arm, Praxis-1 and FLUX 3 Action were compared after fine-tuning on DROID data. Both models were given the same ten tasks, including putting a banana on a plate and closing a drawer. Each task was repeated ten times: Praxis-1 succeeded 88 times, while FLUX 3 Action succeeded 80 times.

Runway also tested whether the model could be transferred to a type of robot it had not encountered before. For this, the company used data from a mobile platform with a gripper arm that Google had previously used to develop RT-1. These recordings were not included in earlier stages of Praxis-1’s training. After fine-tuning on about 95 hours of data, the robot achieved an average success rate of 80% in simulation. The test covered four types of tasks, including grasping soda cans, opening and closing drawers, and placing an apple in a box. Google’s original RT-1 model scored 52.4% on the same benchmark.

Access and limitations

Developers can download Praxis-1 from Hugging Face after registering and agreeing to the access terms. The code is published on GitHub. Runway says versions pre-adapted for different robots are already available alongside the base model. The base model itself must be fine-tuned before use with a new system.

The standard version requires a GPU with 80 GB of memory. According to Runway’s estimates, the compressed version should fit on an RTX 5090 with 32 GB of video memory. However, peak memory usage has not yet been measured; the GitHub documentation notes this.

According to Runway’s research report, Praxis-1 sometimes confuses similar objects and struggles with vague instructions. For long tasks, individual steps must be spelled out: the model cannot yet reliably plan complex sequences of actions on its own.

Runway sees Praxis-1 primarily as a system for executing movements quickly. In the future, it could work alongside general-purpose AI models that break complex tasks down into individual steps.

Dataist’s take: The results show that video training can be used to control robots, but adapting to new systems still requires specialized data. The comparisons were also conducted by Runway itself, and the results have not yet been independently verified.