Glossary · Robotics and Physical AI
Robot Learning
139 terms in the Robotics and Physical AI glossary, each with its one-line definition.
139 terms, A to Z
- 3D Diffusion Policy DP3 Robot control method that generates actions from 3D point clouds instead of flat images, learning manipulation from fewer demonstrations.
- Action Chunking with Transformers ACT Imitation learning method that predicts a short sequence of future arm movements at once, giving smooth motion from a few dozen demonstrations.
- Action Horizon Number of predicted action steps a learned controller actually executes before it observes again and replans.
- Action Normalization Rescaling robot actions and sensor readings into a standard numeric range so a neural network trains well and behaves the same at deployment.
- Action Space Set of commands a learned robot controller is allowed to output, such as joint angles, torques, hand motions or gripper open and close.
- Actor-Critic Reinforcement learning design pairing an actor that picks actions with a critic that scores situations, making learning faster and steadier.
- Advantage Function Reinforcement learning score of how much better an action was than the agent's typical action in the same situation.
- Adversarial Motion Priors AMP Training method that rewards legged robots for moving like recorded animal or human motion clips, judged by a learned discriminator.
- Asymmetric Actor-Critic Reinforcement learning setup where the critic sees perfect simulator state while the actor sees only real sensor inputs.
- Behavior Cloning BC Teaching a robot by training a model to copy what a human expert did in each recorded situation, the simplest form of imitation learning.
- Behavior Transformer BeT Imitation learning method that picks a typical action category with a transformer and then refines the exact action within it.
- Bellman Equation Recursive relation stating that a situation's value equals the immediate reward plus the discounted value of what follows.
- Catastrophic Forgetting Tendency of a neural network to lose previously learned skills when it is trained on new data.
- Causal Confusion Imitation learning failure where a robot keys on a cue that merely accompanies the right action rather than its real cause.
- Checkpoint Selection Choosing which saved training snapshot of a model to deploy, usually by testing candidates since the last one is not always best.
- Classifier-Free Guidance CFG Technique that steers a diffusion model more strongly toward its instruction by blending conditioned and unconditioned predictions.
- Compounding Error Snowballing of small mistakes when a robot copies demonstrations, as each slip leads to less familiar states and bigger slips.
- Conditional Variational Autoencoder CVAE Generative neural network that encodes examples into a small hidden code and rebuilds outputs tailored to a given input.
- Conservative Q-Learning CQL Offline reinforcement learning method that deliberately undervalues actions absent from the recorded data so the robot avoids untested moves.
- Consistency Policy Robot policy distilled from a diffusion policy so it produces actions in one or a few steps, fast enough for tight control loops.
- Continual Learning Teaching a model new tasks over time without making it forget the ones it already knows.
- Contrastive Learning Training without hand labels by teaching a model which pairs of examples belong together and which do not.
- Convolutional Neural Network CNN Neural network for images that slides small learned filters across a picture to detect edges, shapes and objects.
- Credit Assignment Problem Challenge of working out which earlier actions deserve credit or blame for an outcome that appears only later.
- Curriculum Learning Training a learning system on easy tasks first and harder ones later so it builds skills step by step.
- Dataset Aggregation DAgger Imitation method where the robot practices, an expert labels what it should have done and those corrections join the training data.
- Deep Deterministic Policy Gradient DDPG Early off-policy deep reinforcement learning method for continuous actions that trains a deterministic actor alongside a critic.
- Deep Learning Machine learning with many-layered neural networks trained on large data, the basis of modern robot perception and learned control.
- Deep Q-Network DQN DeepMind reinforcement learning method that scores each discrete action with a neural network, famous for learning Atari games.
- Deep Reinforcement Learning Deep RL Trial and error learning in which the decision maker is a deep neural network, used to train walking robots in simulation.
- Delta Action Robot command expressed as a change from the current pose, such as move 2 centimeters forward, rather than an absolute target.
- Denoising Diffusion Implicit Model DDIM Faster sampler for diffusion models that takes a few large deterministic steps, letting diffusion policies act in real time.
- Diffusion Policy Imitation learning method that refines random noise step by step into smooth robot action sequences, handling varied demonstrations well.
- Direct Preference Optimization DPO Fine-tuning a model from human preference pairs without training a separate reward model.
- Discount Factor Reinforcement learning number, usually written gamma, that sets how much future rewards count compared with immediate ones.
- Distribution Shift Mismatch between the situations a model saw in training and those it meets in real use, which degrades performance.
- Dreamer Family of reinforcement learning agents that learn a world model and practice in imagined rollouts, needing far less real experience.
- End-to-End Learning Training one model to map raw sensor input directly to actions instead of chaining separately engineered modules.
- Entropy Regularization Rewarding a learning agent for keeping randomness in its choices so it keeps exploring rather than settling too early.
- Episode One complete attempt at a task from start until success, failure or timeout, the basic unit of robot data and practice.
- Epsilon-Greedy Policy Exploration rule where an agent usually takes its best known action but occasionally picks a random one.
- Evaluation Protocol Fixed rules for testing a robot policy, covering scenes, object placements, trial counts, time limits and success criteria.
- Exploration A learning agent trying new actions to discover better behavior instead of always repeating what already works.
- Exponential Moving Average Weights EMA Slowly updated smoothed copy of model weights that is usually deployed because it behaves more steadily.
- Feature-wise Linear Modulation FiLM Technique that lets one input, such as an instruction, scale and shift each feature channel of a network processing another input.
- Few-Shot Learning Learning a new task from only a handful of examples, often one to ten demonstrations.
- Fine-Tuning Further training of a pretrained model on a small specific dataset to adapt it to a robot, place or skill.
- Gaussian Mixture Model GMM Probability model that blends several bell curves, letting a robot policy represent more than one valid action.
- Generalization A model's ability to handle situations unlike its training data, such as new objects, rooms, instructions or robots.
- Generalized Advantage Estimation GAE Standard reinforcement learning formula that blends short and long-term evidence to estimate an action's advantage.
- Generative Adversarial Imitation Learning GAIL Imitation method where the robot is rewarded for behavior a discriminator cannot tell apart from expert demonstrations.
- Goal-Conditioned Policy Robot controller that takes a goal, such as a target image or instruction, as input so one model serves many goals.
- Gradient Descent Basic neural network training method that repeatedly nudges parameters in the direction that most reduces error.
- Group Relative Policy Optimization GRPO Critic-free reinforcement learning method that scores each attempt against the average of a group of attempts.
- Hierarchical Reinforcement Learning HRL Reinforcement learning where a high-level policy picks goals or skills and lower-level policies execute them.
- Hindsight Experience Replay HER Reinforcement learning trick that relabels a failed attempt as success at wherever the robot actually ended up.
- Human-in-the-Loop Sample-Efficient Robotic Reinforcement Learning HIL-SERL UC Berkeley system that teaches real robots precise skills in hours using demonstrations, human corrections and a learned success judge.
- Imitation Learning IL Teaching a robot by showing it how a task is done, usually through teleoperated recordings, rather than by trial and error.
- Implicit Behavior Cloning IBC Imitation method that learns to score candidate actions and then searches for the highest scoring one.
- Implicit Q-Learning IQL Offline reinforcement learning method that improves skills from recorded data without ever querying actions absent from it.
- Interactive Imitation Learning Imitation learning in which a human watches the robot work and adds corrections or takeovers to the training data.
- Intrinsic Motivation Rewards an agent gives itself for finding something new or surprising, helping it explore when task rewards are rare.
- Inverse Reinforcement Learning IRL Inferring the reward an expert seems to pursue from demonstrations, then training a robot to pursue it.
- Keyframe Prediction Control approach where a model predicts only key gripper poses and a motion planner fills in the path between them.
- Knowledge Distillation Training a smaller student model to reproduce a larger teacher's outputs so it runs faster and cheaper.
- Locomotion Policy Learned controller that makes a legged robot walk, run, climb stairs or recover from falls, usually trained in simulation.
- Long-Horizon Task Robot job spanning many steps over several minutes, such as clearing a table or folding laundry.
- Loss Function Number measuring how wrong a model's predictions are, which training works to minimize.
- Low-Rank Adaptation LoRA Cheap fine-tuning method that freezes a large model and trains only small add-on matrices.
- Machine Learning ML Methods that let computers improve at a task by finding patterns in data instead of following hand-written rules.
- Markov Decision Process MDP Standard mathematical model of a decision problem in reinforcement learning, built from states, actions, transitions and rewards.
- Meta-Learning Learning how to learn, by training across many tasks so a model adapts to a new one from very little data.
- Model-Based Reinforcement Learning MBRL Reinforcement learning where the agent learns or is given a world model and plans or practices inside it.
- Model-Free Reinforcement Learning Reinforcement learning purely from experience without an explicit world model, simple and robust but data hungry.
- Monte Carlo Methods MC Reinforcement learning methods that estimate values by averaging total rewards observed over complete episodes.
- Motion Tracking Policy Humanoid controller trained in simulation to follow recorded human motions, such as dances, while staying balanced.
- Multi-Agent Reinforcement Learning MARL Reinforcement learning with several agents learning together in a shared environment, cooperating, competing or both.
- Multi-Task Learning Training one model on many tasks at once so learning for one helps the others.
- Multilayer Perceptron MLP Simplest deep neural network, with fully connected layers, often used as a fast walking controller for legged robots.
- Multimodal Action Distribution Case where several quite different actions are all correct at the same moment, which naive averaging models get wrong.
- Neural Network Computer model of layered connected units whose connection strengths are adjusted in training to map inputs to outputs.
- Observation History Short window of recent sensor readings or frames given to a robot controller as short-term memory.
- Observation Space Everything a robot controller receives as input, such as camera images, joint positions, forces and instructions.
- Off-Policy Learning Reinforcement learning that can learn from experience gathered by any source, including old versions of itself or humans.
- Offline Reinforcement Learning Offline RL Improving robot behavior using only a fixed set of previously recorded experience, with no further practice.
- On-Policy Learning Reinforcement learning that updates only with fresh experience from the current behavior, discarding older data.
- Online Reinforcement Learning Reinforcement learning in which the agent keeps acting and learning from fresh experience rather than a fixed recording.
- Out-of-Distribution OOD Describes inputs outside the range of situations a model saw in training, such as unfamiliar objects or lighting.
- Overfitting When a model memorizes training examples too closely, including irrelevant details, and performs poorly on anything new.
- Partially Observable Markov Decision Process POMDP Reinforcement learning problem in which the agent cannot see the full situation and must act on incomplete information.
- Perceiver-Actor PerAct Instruction-following manipulation model that voxelizes a 3D workspace view and predicts the next gripper pose.
- Policy Decision-making function that turns what a robot senses into what it does next, from simple rules to large AI models.
- Policy Distillation Compressing one or more trained robot controllers into a single, often smaller, controller trained to copy them.
- Policy Evaluation Measuring how well a trained robot controller performs in simulation, on hardware or in a learned world model.
- Policy Gradient Reinforcement learning methods that directly adjust a policy toward behavior that earned higher reward in trial runs.
- Privileged Information Data available during simulated training but not on the real robot, such as exact friction or object positions.
- Proximal Policy Optimization PPO Most widely used reinforcement learning algorithm for simulated robot training, improving the policy in small clipped steps.
- Q-Function Reinforcement learning estimate of total future reward from taking a given action in a given situation.
- Q-Learning Classic off-policy reinforcement learning method that updates action values toward the best estimated next action value.
- Rapid Motor Adaptation RMA Method that lets a walking robot infer terrain or load changes from recent joint readings and adjust its gait instantly.
- Real-World Reinforcement Learning A physical robot learning by trial and error in the real world rather than in a simulator.
- Recurrent Neural Network RNN Neural network that processes sequences step by step while carrying a running memory forward.
- REINFORCE Simplest policy gradient algorithm, making actions that led to high total reward more likely.
- Reinforcement Learning RL Learning by trial and error, where an agent tries actions, receives rewards or penalties and adjusts its behavior.
- Reinforcement Learning from Human Feedback RLHF Training an AI from human ratings or comparisons of its outputs, then optimizing it to produce more of what people preferred.
- Replay Buffer Memory store of a learner's past experience that it can sample and study again many times.
- Representation Learning Teaching a model to turn raw input such as pixels into a compact, meaningful summary that makes later decisions easier.
- Residual Learning Learning a small correction added on top of an existing controller rather than learning the whole behavior.
- Return Total reward an agent collects from a moment until the episode ends, usually with later rewards discounted.
- Reward Function Scoring rule that tells a reinforcement learning agent how well it is doing at each step.
- Reward Hacking A learning agent exploiting an unintended loophole to earn a high score without doing the intended task.
- Reward Model Learned model that estimates how well a robot is doing or how close it is to finishing a task.
- Reward Shaping Adding intermediate rewards that guide a learning agent toward a goal and speed up training.
- Robot Learning Field that uses machine learning to give robots skills from data and experience instead of hand programming every motion.
- Rollout Running a robot controller in a real or simulated environment for one or more attempts to gather data or test it.
- Safe Reinforcement Learning Safe RL Reinforcement learning that must respect safety limits during training or deployment, such as force or zone constraints.
- Sample Efficiency How much data or practice a learning method needs to reach a given level of skill.
- SARSA On-policy reinforcement learning algorithm that learns action values from the actions its current behavior actually takes.
- Self-Supervised Learning SSL Training on raw unlabeled data by solving puzzles built from the data itself, such as predicting hidden image parts.
- Soft Actor-Critic SAC Off-policy maximum-entropy reinforcement learning algorithm for continuous control, popular for learning on real robots.
- Sparse Reward Reward given only when a task is fully completed, easy to define but hard to learn from.
- Success Detector Learned model that judges from images or sensor data whether a robot completed its task, replacing a human judge.
- Success Rate Share of test attempts in which a robot completes its task, the most common headline number in robot learning.
- Supervised Fine-Tuning SFT Further training of a pretrained model on labeled examples of the wanted behavior, such as task demonstrations.
- Target Network Slowly updated copy of a network used to compute stable learning targets in deep reinforcement learning.
- Task Progress Score Test score giving partial credit for each completed stage of a task rather than only full success or failure.
- Teacher-Student Learning Two-stage training where a teacher learns with perfect simulator information and a student imitates it using real sensors.
- Temporal Difference Learning TD Learning value estimates step by step by updating toward the reward received plus the estimated value of the next state.
- Temporal Ensembling Smoothing trick that blends overlapping predicted action chunks for the current moment to avoid jerky handovers.
- Training and Inference The learning phase of a model on data and the usage phase when the finished model makes decisions, often onboard.
- Transfer Learning Reusing what a model learned on one task or data type to get a head start on another.
- Trust Region Policy Optimization TRPO Policy gradient method that limits how much the policy changes per update for stability, a forerunner of PPO.
- Twin Delayed Deep Deterministic Policy Gradient TD3 Continuous-control reinforcement learning algorithm that fixes DDPG's instability with twin critics and delayed updates.
- U-Net U-shaped neural network that compresses then expands its input with skip links, used in image generators and some action generators.
- Value Function Estimate of the total future reward an agent can expect starting from a given situation.
- Value Iteration Classic planning algorithm that repeatedly applies the Bellman optimality equation until state values stop changing.
- Variational Autoencoder VAE Neural network that compresses data into a smooth latent space and reconstructs it, so new examples can be sampled.
- Visuomotor Policy Robot policy that maps camera images, usually with joint readings, directly to motor commands.
- Zero-Shot Generalization Handling a task, object or place never seen in training without extra examples or retraining.
Full entry for members
Every term has a full entry for members: the plain-words definition, the practitioner note and the typed links between terms.
ACW Robotics and Physical AI Glossary, edition 2026.11.1. The headword and one line are open to everyone.