Will video games train the robots of tomorrow?

Video games are no longer just an entertainment market, and could become one of the most strategic resources of the next generation of artificial intelligence. While large language models have learned from web data, robots, autonomous vehicles and future physical agents need a different type of learning: understanding how their actions transform their environment. Virtual worlds offer precisely this field of experimentation.

The raising of 8 million euros from the British startup Worldmodeldata illustrates this trend. The company is not developing a language model, robot or electronic chip, but is building a library of training data from video games in order to power “world models”, these models capable not only of understanding the world, but of acting in it. An innovation that could be transformative for a sector that is very challenged today. Thus, video games could become a strategic infrastructure for physical AI, and find a second economic model based on the valorization of their virtual universes.

After the Internet, where to find the next AI raw material

Generative artificial intelligence was built on an abundant resource captured on the Internet. Large language models have learned to predict the next word by absorbing billions of web pages, books, lines of code and images. This strategy has enabled spectacular progress, but it reaches its limits when the objective is no longer to produce text, but to interact with the world.

Recognizing a hammer in a photograph is one thing, knowing how to grasp it, adapting your grip to its weight, hitting a nail or anticipating the trajectory of a moving object is another. Future robots, autonomous vehicles or industrial agents will have to think in terms of actions, consequences and physical dynamics. They will have to learn causality.

This is precisely the ambition of world models. Unlike generative models, which predict the most probable content, they seek to anticipate the evolution of an environment based on a given action. If a robot pushes a door, what will happen? If a vehicle brakes on a wet road, how will its environment change? This ability to mentally simulate the consequences of a decision constitutes one of the main areas of research in AI laboratories.

Why video games are now of interest to researchers

To train these models, the data available on the Internet is insufficient. A video shows a car turning left. It does not say why, with what steering angle, what tire grip, what speed or what forces are exerted on the vehicle.

Conversely, a video game engine knows every variable in the simulation. It calculates gravity, collisions, speeds, masses, trajectories, mechanical constraints and all interactions between objects. Every player action is recorded with its context and consequences.

This difference is fundamental, the engines developed by Epic Games with Unreal Engine or by Unity Technologies do not simply produce images, but generate fully simulated environments of which each state is known. For a laboratory working on robotics or autonomous vehicles, these worlds constitute a source of data of unprecedented richness.

This is precisely the economic model of Worldmodeldata. The startup transforms games played on titles developed under Unreal or Unity into structured data games intended for AI laboratories. Unlike the mass collection practices that accompanied the rise of LLMs, the company claims to operate exclusively through licensing agreements with studios and creators, opening up the possibility of remuneration for content holders.

Game engines become critical infrastructure

This development goes far beyond just the video game sector; for several years, technologies from video games have been gradually migrating towards the industry.

The CARLA project, developed on Unreal Engine, has become one of the main simulation platforms used by researchers working on autonomous vehicles. Microsoft designed AirSim, also based on Unreal, to train drones and self-driving cars in virtual environments before any real deployment.

For its part, NVIDIA has considerably expanded the ambition of its Omniverse platform. Initially designed for industrial digital twins, it is now used to train robots via Isaac Sim, a simulation environment based on the same principles as game engines: faithfully reproducing the laws of physics in order to allow machines to learn without risk.

Same logic at Google DeepMind with Genie and Genie 2, capable of generating interactive worlds from images, or at Meta, which is developing Habitat to train incarnated agents in virtual environments.

The goal is to replace part of the learning in the real world with billions of iterations carried out in simulation.

The economic stakes are immense, because training a robot in a factory requires expensive equipment, immobilizes equipment and involves risks, but training thousands of robots simultaneously in a virtual universe only requires computing resources.

Publishers discover a second economic model

This change could profoundly transform the video game industry itself. Until now, the economic value of a game was based on sales, subscriptions, additional content or microtransactions. Tomorrow, virtual universes could also become strategic assets for training artificial intelligence.

Studios already have what labs are looking for: cities, roads, buildings, characters, human behavior, physical interactions and millions of hours of gameplay.

Their heritage would no longer be limited to intellectual property intended for entertainment, and would become an exploitable resource to train the robots of tomorrow.

Worldmodeldata is precisely trying to organize this new value chain. Its ambition is to build a library of more than a million hours of data by the end of 2026, compared to around 40,000 hours for the largest sets available today. If this promise is kept, the company would have a strategic asset that is difficult to replicate.

This prospect could open up an unprecedented licensing market between game publishers and AI laboratories. As music platforms have learned to monetize their catalogs with streaming services, studios could tomorrow promote their universes to developers of autonomous systems.

A new AI value chain is emerging

Worldmodeldata is not an isolated case, several companies are already building the different bricks of this new economy.

Encord develops platforms to annotate and manage data for physical AI. Scale AI continues its expansion into data used by the defense, robotics and autonomous vehicle sectors. Labelbox is also positioned on the organization of industrial data sets.

In the world model field, the competition is accelerating. World Labs, founded by Fei-Fei Li, aims to build models capable of understanding and generating coherent three-dimensional environments. Stanhope AI works on adaptive models for robotics and defense. Physical Intelligence is developing a foundation model intended to control robots from different manufacturers. Skild AI pursues a similar ambition with a universal platform for embedded AI. BeyondMath applies these approaches to physics simulation.

These companies do not directly build the robots of the future, but develop the software infrastructure, data and training environments that will then allow other players to design them.

The limits of simulation

However, the idea that robots could learn only in video games remains excessive. Robotics has long been faced with the problem of sim-to-real gap. A system that performs well in a simulation can fail in its first real deployment. Materials age, sensors drift, human behavior is unpredictable, and industrial environments are rarely as clean as their digital counterparts.

Video games also introduce their own biases. Players take risks that no real driver would accept. Physics engines sometimes favor the fluidity of the experience to the detriment of absolute scientific fidelity. Finally, rare events, although essential for training critical systems, remain difficult to reproduce.

Simulation will therefore not replace data from the field, but will, however, significantly reduce the cost and duration of the learning phases before validation in real conditions.

The next AI battle will be fought over data

For three years, the global competition has focused on language models, semiconductors and computing capabilities. A new layer of value now appears, that of data allowing artificial intelligence to interact with its environment.

This development opens up an unexpected opportunity for the video game industry. Engines developed to create immersive worlds could become critical infrastructures for physical AI. Publishers’ catalogs could acquire value well beyond entertainment. And the data produced by millions of players could become one of the most sought-after resources by laboratories working on robots, autonomous vehicles or industrial agents.

If the great language models are the children of the Internet, artificial intelligences capable of acting in the real world could well be, tomorrow, the children of virtual worlds.