Real2Sim2Real: when simulation becomes the heart of physical AI

Artificial intelligence, as we know it today, developed in an essentially abstract universe. Text data, images, digital signals: AI is progressing in a world where errors are corrected by software iteration. Its entry into the physical world, whether it involves autonomous vehicles, industrial robots, or humanoids, profoundly modifies the rules of the game. Here, failure no longer results in a drop in performance, but in a material, even human, risk. In this context, simulation changes status and goes from a support tool to a central infrastructure.

The acquisition this week of Mentee Robotics by Mobileye Global perfectly illustrates this change, and highlights the Real2Sim2Real. Behind this abstruse term lies a transformation in the way physical AI is designed, trained and brought to industrial scale.

Reality, a structurally limited learning ground

In the physical world, data is neither abundant nor free, critical situations are rare, environments change, and human behavior is difficult to model. Testing an autonomous driving algorithm or a manipulating humanoid in real conditions involves high costs and unavoidable risks. The higher the system aims for a high level of reliability, the less it is possible to rely exclusively on direct experimentation.

This paradox is at the heart of the thinking of those involved in advanced robotics. Whether it is autonomous driving or humanoid manipulation, the central problem is no longer to achieve an acceptable average performance, but to manage improbable edge cases, often absent from real datasets.

From simulation as a support to simulation as a basis

The Real2Sim2Real pipeline formalizes an evolution where reality is no longer the main learning area, but an anchor and validation point. Data from the physical world feeds the simulation; this then becomes the privileged space for training, stress testing and generalization. Finally, the skills acquired are transferred to reality, with continuous correction loops.

This approach is found, in different forms, among actors like NVIDIAwhose robotic simulation environments have become standard bricks for training physical agents, or at Figure AIwhich relies on massive volumes of synthetic scenarios to prepare its humanoids for real industrial environments. Simulation does not seek to faithfully reproduce the world, but to cover a sufficiently wide diversity of situations to allow generalization.

Generalization and rapid learning: a shared issue

In humanoid robotics, the question of versatility is central, so a useful robot cannot be limited to a single task, repeated in a fixed environment and must learn quickly, adapt, recombine existing skills. At Mentee Robotics, this translates into work on the few-shot learningwhere a few human demonstrations are enough to teach a new action.

This logic is found among other emerging players in “physical AI”, such as 1X Technologieswhich targets semi-structured environments, or Skild AIwhich develops generic models for robots capable of transferring skills between platforms.

Security, auditability and simulation as a proof tool

As AI approaches the human body, the question of security becomes structuring. It is no longer just a matter of demonstrating that a system works, but of proving that it behaves in a predictable, responsible and fully verifiable manner. In the automobile industry, Mobileye has built its credibility on formal models of decision and responsibility; humanoid robotics is today at a stage comparable to that of autonomous driving around ten years ago.

Simulation offers here an irreplaceable field of experimentation, and makes it possible to confront systems with extreme situations, to evaluate risky behavior and to test security frameworks without real exposure. For players aiming for large-scale industrial deployments, this capacity becomes a prerequisite rather than a competitive advantage.

An invisible, but decisive infrastructure

The amounts committed by players in the sector give an order of magnitude of the issues. Startups like Figure AIwhich has raised more than $1.7 billion, Skild AIwith 300 million dollars in its first round, or Neura Roboticsfunded to the tune of approximately $260 million cumulatively, are investing massively in simulation environments capable of generating millions, even tens of millions of synthetic scenarios. Added to these volumes are ongoing computing needs, which several players believe represent annual budgets of several tens of millions of dollarsonly for training and validation.

This invisible layer brings physical AI closer to classic industrial logic, far from the low marginal cost software paradigm. Development cycles take place over several years, while initial investments, combining simulation, calculation, hardware engineering and software integration, can reach several hundred million dollars before any significant deployment.