The sim-to-real gap is closing fast. How foundation models are teaching robots to understand the physical world.
Robotics has spent forty years as the field where software promises outran hardware reality, but the arrival of foundation models trained on physics simulation, robotic interaction logs, and web-scale video is finally closing that gap, teaching machines to generalize across tasks the way large language models generalize across text.
The convergence of language and control
The core technical shift is that vision-language models and robotic control policies have merged into a single architecture. Systems like Google DeepMind's RT-3 and NVIDIA's GR00T take a plain-language instruction such as pick up the red cup and place it on the shelf, combine it with a live camera feed, and output motor torques or joint targets directly, with no task-specific code path in between. Earlier generations of robots needed a separate program for every task variation; a foundation model instead treats picking, placing, folding, and assembling as different prompts to the same underlying policy.
This works because the model has learned a shared representation of objects, affordances, and physical consequences from an enormous and varied training mixture. It has seen thousands of hours of teleoperated manipulation, millions of simulated physics rollouts, and web video of humans handling everyday objects. That breadth is what lets it improvise on a task it was never explicitly trained on, in the same way a language model can answer a question it never saw verbatim during training.
Closing the sim-to-real gap
The sim-to-real transfer gap has historically been robotics' hardest unsolved problem: a policy trained entirely in simulation would fall apart the moment it met real friction, real lighting, and real sensor noise. Two advances have narrowed that gap dramatically. Physics simulators are now far more accurate at modeling contact dynamics, deformable materials, and sensor imperfections, and domain-randomization techniques deliberately vary textures, lighting, and physical parameters during training so the resulting policy is robust to the mismatch between simulation and the real world.
The numbers reflect the shift. RT-3 style models trained entirely in simulation now reach roughly 78 percent success rates on real-world manipulation benchmarks, up from about 35 percent just two years ago. That is the difference between a research demo and something a warehouse operator would actually deploy on a production line.
From lab demo to factory floor
The commercial rollout is already substantial. Amazon has deployed more than 750,000 AI-powered robots across its fulfillment network, covering picking, packing, sorting, and quality inspection, with foundation-model-driven perception replacing many of the hard-coded vision pipelines used in earlier robot generations. Tesla's Optimus humanoid, running on a custom foundation model, is now handling repetitive assembly tasks on two production lines at the Fremont factory, a step beyond the choreographed demo videos that first introduced the platform.
These deployments matter because they generate exactly the kind of real-world interaction data that improves the next generation of models. Every successful grasp, every recovered failure, and every edge case a robot encounters on a real line becomes training signal, creating a flywheel that a simulation-only approach could never match on its own.
What is still unsolved
None of this means general-purpose robotics has arrived. Long-horizon tasks that require planning dozens of steps ahead, manipulation of genuinely novel objects with unfamiliar physical properties, and safe operation around humans in unstructured environments remain hard. Foundation models are also expensive to run at the low latency robotic control requires, which is pushing labs toward distillation and on-device inference rather than shipping every decision to a cloud model.
The research literature reflects how fast this space is moving, with new robotics foundation model papers appearing on a near-weekly basis across labs, and reproducing or even just tracking every relevant result has become a job in itself. Vincony's Deep Research tool has become a popular resource for exactly this problem, letting researchers pull key findings, benchmark numbers, and methodology comparisons out of hundreds of papers in a single session rather than reading each one individually.
The trajectory is clear even where the destination is not: robots are moving from single-purpose machines running bespoke code toward general-purpose bodies running a shared foundation model, mirroring the path large language models took from narrow chatbots to general assistants. The sim-to-real gap will not close to zero, but it no longer has to, because it has narrowed enough for foundation-model robots to already be doing real economic work.