AutomotiveCloudWatch

Glossary · Robotics and Physical AI

Simulation and Sim-to-Real

76 terms in the Robotics and Physical AI glossary, each with its one-line definition.

76 terms, A to Z

  • Actuator Network Small neural network trained on real motor data that replaces an idealized motor model in simulation to narrow the sim-to-real gap.
  • AI2-THOR Interactive household simulator from the Allen Institute for AI where virtual agents open, pick up and move objects in rooms.
  • Automatic Domain Randomization ADR Domain randomization that widens its ranges automatically as the policy improves, creating a curriculum for robust sim-to-real transfer.
  • BEHAVIOR-1K Stanford benchmark of 1,000 everyday household activities simulated in realistic virtual homes to test long-horizon robot AI.
  • CALVIN Simulated benchmark scoring how many chained language instructions a robot arm policy can complete in a row.
  • CARLA Open-source Unreal Engine simulator for autonomous driving research with towns, traffic, weather and simulated cameras and lidar.
  • Choreonoid Open-source robot simulator and motion choreography tool from Japan's AIST, used for humanoid and disaster response research.
  • Collision Mesh Simplified 3D shape used by simulators and planners for contact checks, separate from the detailed shape used for display.
  • Contact Modeling How a simulator detects touches and computes push-back, sliding and bouncing between objects, a main source of sim-to-real error.
  • Convex Decomposition Splitting a complex shape into simple convex pieces so physics engines can check collisions quickly.
  • CoppeliaSim Desktop robot simulator for research and teaching that models arms, mobile robots and work cells scripted in Lua or Python.
  • Deformable Object Simulation Simulating things that bend, stretch or flow, such as cloth, rope, cables and liquids, which is much harder than solid objects.
  • Differentiable Simulation Simulator that also reports how outcomes change with each input, letting gradients tune motions, controllers or physical parameters.
  • Digital Twin Virtual copy of a robot, work cell or factory kept in step with the real thing for planning, testing and training.
  • Domain Adaptation Adjusting an AI model trained in one setting, such as simulation, so it works in another, such as the real world.
  • Domain Randomization DR Training across many deliberately varied simulations so the real world looks like just one more variation to the policy.
  • Drake Toolbox for modeling, simulating and optimizing robots with accurate contact physics, led by Toyota Research Institute.
  • Dynamics Randomization Varying simulated physical properties such as mass, friction, motor strength and delay during training for robust real-world transfer.
  • Friction Model Rule a simulator uses to decide how much touching surfaces resist sliding, often randomized during training.
  • Gaussian Splatting 3DGS Photo-based 3D reconstruction that stores a scene as millions of soft colored blobs, viewable from new angles in real time.
  • Gazebo Free open-source robot simulator closely tied to ROS, a common first simulator for testing navigation and sensors.
  • Generative Simulation Using generative AI to create simulation content such as 3D objects, layouts, tasks and reward code automatically.
  • Genesis Open-source Python physics platform that simulates rigid, soft and fluid materials together and runs fast on GPUs for robot learning.
  • GPU-Parallel Simulation Running thousands of simulation copies at once on one GPU so robot policies get massive practice in minutes or hours.
  • Gymnasium Standard Python interface for reinforcement learning environments and the maintained successor to OpenAI Gym.
  • Habitat Meta simulation platform for training AI agents to navigate and rearrange objects in realistic 3D scans of real homes.
  • Hardware-in-the-Loop HIL Testing real robot computers or controllers against a simulated robot and world to catch timing and wiring issues safely.
  • Hydroelastic Contact Contact model that treats touching bodies as slightly soft and spreads pressure over a patch for smoother grasp forces.
  • Isaac Gym Former NVIDIA tool that ran physics and RL training together on a GPU for thousands of robots, now replaced by Isaac Lab.
  • Isaac Lab NVIDIA's open-source framework on Isaac Sim for training robot policies in thousands of parallel simulations.
  • Isaac Sim NVIDIA's robot simulator built on Omniverse with accurate physics, photoreal rendering and simulated sensors.
  • Legged Gym Training framework from ETH Zurich showing quadrupeds could learn to walk in simulation in minutes on one GPU.
  • LIBERO Simulated manipulation benchmark for lifelong robot learning, now a standard scorecard for vision-language-action models.
  • ManiSkill GPU-parallel simulator and benchmark from UC San Diego for training general robot manipulation policies.
  • Material Point Method MPM Simulation technique mixing particles and a grid for materials that deform, flow or break, such as dough, sand and fluids.
  • Meta-World Simulated benchmark of 50 robot arm tasks used to test multi-task and meta-learning methods.
  • MuJoCo Free physics engine known for fast, accurate contact, a standard tool in robot learning research.
  • MuJoCo Playground Collection of ready-to-train GPU robot tasks on MuJoCo for quadrupeds, humanoids and hands, with real-hardware transfer.
  • MuJoCo Warp MJWarp MuJoCo rewritten for massively parallel simulation on NVIDIA GPUs, developed by Google DeepMind and NVIDIA.
  • MuJoCo XLA MJX MuJoCo rewritten in JAX to run huge batches of simulations on GPUs and TPUs, mainly for fast locomotion training.
  • Neural Radiance Field NeRF AI method that learns a 3D scene from photos and renders realistic new views, stored inside a neural network.
  • Newton Physics Engine Newton Open-source GPU physics engine for robot learning from NVIDIA, Google DeepMind and Disney Research, compatible with MuJoCo and Isaac Lab.
  • NVIDIA Warp Python framework that compiles ordinary-looking Python into fast GPU code for physics, cloth and particle simulation.
  • Objaverse Large free collection of 3D object models gathered from the web, used to fill robot simulation scenes with variety.
  • Observation Noise Random error added on purpose to simulated sensor readings during training so policies do not rely on perfect signals.
  • OmniGibson Stanford household robot simulator on NVIDIA Omniverse that models cloth, liquids and object states, behind BEHAVIOR-1K.
  • Omniverse NVIDIA platform for building 3D simulation and digital twin applications on OpenUSD, underlying Isaac Sim.
  • Omniverse Replicator NVIDIA tool that randomizes and renders scenes to create perfectly labeled synthetic training images.
  • Physics Engine Software that steps object motion under gravity, collisions, friction and joints, the core of every robot simulator.
  • PhysX Real-time physics engine owned by NVIDIA that started in games and now powers GPU robot simulation in Isaac tools.
  • Procedural Scene Generation Automatically creating many varied simulated rooms or environments with rules or AI instead of hand modeling.
  • PyBullet Python interface to the Bullet physics engine, popular for robot RL research in the late 2010s and still used for prototypes.
  • Real-to-Sim Building a simulation from real-world captures such as scanned rooms, reconstructed objects and measured robot behavior.
  • Real-to-Sim-to-Real Workflow that copies a real scene into simulation, trains or tunes a policy there and deploys it back on the real robot.
  • Rigid-Body Simulation Simulating objects as perfectly solid bodies linked by joints, fast for arms and walkers but poor for cloth or soft items.
  • RLBench Simulated benchmark of 100 robot arm tasks used to compare learning methods, especially those using 3D views.
  • RoboCasa Large simulation framework of virtual kitchens with benchmark tasks and datasets for household robot learning.
  • Robosuite MuJoCo-based simulation framework with standard robot arm models and tasks, the base of RoboCasa and LIBERO.
  • Robot Simulation Running a virtual robot and its surroundings on a computer to develop, train and test software without hardware.
  • SAPIEN UC San Diego simulator for objects with moving parts, paired with the PartNet-Mobility dataset and the base of ManiSkill.
  • Sensor Simulation Producing realistic simulated camera, depth, lidar and other readings so perception can be trained before hardware exists.
  • Sim-to-Real Gap Difference between how a robot behaves in simulation and in reality, a central problem in robot learning.
  • Sim-to-Real Transfer Sim2Real Getting a policy or model trained in simulation to work on a real robot, as modern walking robots do.
  • Sim-to-Sim Transfer Checking a policy trained in one simulator inside another, such as Isaac Lab to MuJoCo, before trying hardware.
  • SimplerEnv Simulated test scenes that mirror real robot setups so simulation scores predict real-world policy performance.
  • SimReady Asset 3D object prepared for simulation with mass, collision shapes, joints, materials and labels so it behaves realistically.
  • Simulation in Continuous Integration Running simulated test scenarios automatically on every code change to catch regressions before they reach robots.
  • Simulation Timestep Simulated time per physics step, where smaller steps are more stable and accurate but slower.
  • Software-in-the-Loop SIL Testing a robot's real, unchanged software against a simulated robot and world instead of hardware.
  • Synthetic Data Training data created by simulation, rendering or generative AI instead of recorded in the real world.
  • System Identification SysID Measuring a real robot's physical properties from test data so simulators and controllers match reality.
  • Terrain Curriculum Training plan where simulated walking robots move to harder terrain as they succeed and drop back when they fail.
  • Tiled Rendering Rendering many simulated cameras together as one large image then cutting it up, enabling vision RL at scale.
  • Universal Scene Description USD Open 3D scene format and framework from film animation, now the backbone of NVIDIA simulation and digital twins.
  • Vectorized Environment Running many copies of a learning task in lockstep and returning batched results to speed up reinforcement learning.
  • Webots Free open-source desktop robot simulator with a large library of ready-made robot models, popular in education.

Full entry for members

Every term has a full entry for members: the plain-words definition, the practitioner note and the typed links between terms.

See membership

ACW Robotics and Physical AI Glossary, edition 2026.11.1. The headword and one line are open to everyone.