AutomotiveCloudWatch

Glossary · Robotics and Physical AI

Robot Learning

139 terms in the Robotics and Physical AI glossary, each with its one-line definition.

139 terms, A to Z

  • 3D Diffusion Policy DP3 Robot control method that generates actions from 3D point clouds instead of flat images, learning manipulation from fewer demonstrations.
  • Action Chunking with Transformers ACT Imitation learning method that predicts a short sequence of future arm movements at once, giving smooth motion from a few dozen demonstrations.
  • Action Horizon Number of predicted action steps a learned controller actually executes before it observes again and replans.
  • Action Normalization Rescaling robot actions and sensor readings into a standard numeric range so a neural network trains well and behaves the same at deployment.
  • Action Space Set of commands a learned robot controller is allowed to output, such as joint angles, torques, hand motions or gripper open and close.
  • Actor-Critic Reinforcement learning design pairing an actor that picks actions with a critic that scores situations, making learning faster and steadier.
  • Advantage Function Reinforcement learning score of how much better an action was than the agent's typical action in the same situation.
  • Adversarial Motion Priors AMP Training method that rewards legged robots for moving like recorded animal or human motion clips, judged by a learned discriminator.
  • Asymmetric Actor-Critic Reinforcement learning setup where the critic sees perfect simulator state while the actor sees only real sensor inputs.
  • Behavior Cloning BC Teaching a robot by training a model to copy what a human expert did in each recorded situation, the simplest form of imitation learning.
  • Behavior Transformer BeT Imitation learning method that picks a typical action category with a transformer and then refines the exact action within it.
  • Bellman Equation Recursive relation stating that a situation's value equals the immediate reward plus the discounted value of what follows.
  • Catastrophic Forgetting Tendency of a neural network to lose previously learned skills when it is trained on new data.
  • Causal Confusion Imitation learning failure where a robot keys on a cue that merely accompanies the right action rather than its real cause.
  • Checkpoint Selection Choosing which saved training snapshot of a model to deploy, usually by testing candidates since the last one is not always best.
  • Classifier-Free Guidance CFG Technique that steers a diffusion model more strongly toward its instruction by blending conditioned and unconditioned predictions.
  • Compounding Error Snowballing of small mistakes when a robot copies demonstrations, as each slip leads to less familiar states and bigger slips.
  • Conditional Variational Autoencoder CVAE Generative neural network that encodes examples into a small hidden code and rebuilds outputs tailored to a given input.
  • Conservative Q-Learning CQL Offline reinforcement learning method that deliberately undervalues actions absent from the recorded data so the robot avoids untested moves.
  • Consistency Policy Robot policy distilled from a diffusion policy so it produces actions in one or a few steps, fast enough for tight control loops.
  • Continual Learning Teaching a model new tasks over time without making it forget the ones it already knows.
  • Contrastive Learning Training without hand labels by teaching a model which pairs of examples belong together and which do not.
  • Convolutional Neural Network CNN Neural network for images that slides small learned filters across a picture to detect edges, shapes and objects.
  • Credit Assignment Problem Challenge of working out which earlier actions deserve credit or blame for an outcome that appears only later.
  • Curriculum Learning Training a learning system on easy tasks first and harder ones later so it builds skills step by step.
  • Dataset Aggregation DAgger Imitation method where the robot practices, an expert labels what it should have done and those corrections join the training data.
  • Deep Deterministic Policy Gradient DDPG Early off-policy deep reinforcement learning method for continuous actions that trains a deterministic actor alongside a critic.
  • Deep Learning Machine learning with many-layered neural networks trained on large data, the basis of modern robot perception and learned control.
  • Deep Q-Network DQN DeepMind reinforcement learning method that scores each discrete action with a neural network, famous for learning Atari games.
  • Deep Reinforcement Learning Deep RL Trial and error learning in which the decision maker is a deep neural network, used to train walking robots in simulation.
  • Delta Action Robot command expressed as a change from the current pose, such as move 2 centimeters forward, rather than an absolute target.
  • Denoising Diffusion Implicit Model DDIM Faster sampler for diffusion models that takes a few large deterministic steps, letting diffusion policies act in real time.
  • Diffusion Policy Imitation learning method that refines random noise step by step into smooth robot action sequences, handling varied demonstrations well.
  • Direct Preference Optimization DPO Fine-tuning a model from human preference pairs without training a separate reward model.
  • Discount Factor Reinforcement learning number, usually written gamma, that sets how much future rewards count compared with immediate ones.
  • Distribution Shift Mismatch between the situations a model saw in training and those it meets in real use, which degrades performance.
  • Dreamer Family of reinforcement learning agents that learn a world model and practice in imagined rollouts, needing far less real experience.
  • End-to-End Learning Training one model to map raw sensor input directly to actions instead of chaining separately engineered modules.
  • Entropy Regularization Rewarding a learning agent for keeping randomness in its choices so it keeps exploring rather than settling too early.
  • Episode One complete attempt at a task from start until success, failure or timeout, the basic unit of robot data and practice.
  • Epsilon-Greedy Policy Exploration rule where an agent usually takes its best known action but occasionally picks a random one.
  • Evaluation Protocol Fixed rules for testing a robot policy, covering scenes, object placements, trial counts, time limits and success criteria.
  • Exploration A learning agent trying new actions to discover better behavior instead of always repeating what already works.
  • Exponential Moving Average Weights EMA Slowly updated smoothed copy of model weights that is usually deployed because it behaves more steadily.
  • Feature-wise Linear Modulation FiLM Technique that lets one input, such as an instruction, scale and shift each feature channel of a network processing another input.
  • Few-Shot Learning Learning a new task from only a handful of examples, often one to ten demonstrations.
  • Fine-Tuning Further training of a pretrained model on a small specific dataset to adapt it to a robot, place or skill.
  • Gaussian Mixture Model GMM Probability model that blends several bell curves, letting a robot policy represent more than one valid action.
  • Generalization A model's ability to handle situations unlike its training data, such as new objects, rooms, instructions or robots.
  • Generalized Advantage Estimation GAE Standard reinforcement learning formula that blends short and long-term evidence to estimate an action's advantage.
  • Generative Adversarial Imitation Learning GAIL Imitation method where the robot is rewarded for behavior a discriminator cannot tell apart from expert demonstrations.
  • Goal-Conditioned Policy Robot controller that takes a goal, such as a target image or instruction, as input so one model serves many goals.
  • Gradient Descent Basic neural network training method that repeatedly nudges parameters in the direction that most reduces error.
  • Group Relative Policy Optimization GRPO Critic-free reinforcement learning method that scores each attempt against the average of a group of attempts.
  • Hierarchical Reinforcement Learning HRL Reinforcement learning where a high-level policy picks goals or skills and lower-level policies execute them.
  • Hindsight Experience Replay HER Reinforcement learning trick that relabels a failed attempt as success at wherever the robot actually ended up.
  • Human-in-the-Loop Sample-Efficient Robotic Reinforcement Learning HIL-SERL UC Berkeley system that teaches real robots precise skills in hours using demonstrations, human corrections and a learned success judge.
  • Imitation Learning IL Teaching a robot by showing it how a task is done, usually through teleoperated recordings, rather than by trial and error.
  • Implicit Behavior Cloning IBC Imitation method that learns to score candidate actions and then searches for the highest scoring one.
  • Implicit Q-Learning IQL Offline reinforcement learning method that improves skills from recorded data without ever querying actions absent from it.
  • Interactive Imitation Learning Imitation learning in which a human watches the robot work and adds corrections or takeovers to the training data.
  • Intrinsic Motivation Rewards an agent gives itself for finding something new or surprising, helping it explore when task rewards are rare.
  • Inverse Reinforcement Learning IRL Inferring the reward an expert seems to pursue from demonstrations, then training a robot to pursue it.
  • Keyframe Prediction Control approach where a model predicts only key gripper poses and a motion planner fills in the path between them.
  • Knowledge Distillation Training a smaller student model to reproduce a larger teacher's outputs so it runs faster and cheaper.
  • Locomotion Policy Learned controller that makes a legged robot walk, run, climb stairs or recover from falls, usually trained in simulation.
  • Long-Horizon Task Robot job spanning many steps over several minutes, such as clearing a table or folding laundry.
  • Loss Function Number measuring how wrong a model's predictions are, which training works to minimize.
  • Low-Rank Adaptation LoRA Cheap fine-tuning method that freezes a large model and trains only small add-on matrices.
  • Machine Learning ML Methods that let computers improve at a task by finding patterns in data instead of following hand-written rules.
  • Markov Decision Process MDP Standard mathematical model of a decision problem in reinforcement learning, built from states, actions, transitions and rewards.
  • Meta-Learning Learning how to learn, by training across many tasks so a model adapts to a new one from very little data.
  • Model-Based Reinforcement Learning MBRL Reinforcement learning where the agent learns or is given a world model and plans or practices inside it.
  • Model-Free Reinforcement Learning Reinforcement learning purely from experience without an explicit world model, simple and robust but data hungry.
  • Monte Carlo Methods MC Reinforcement learning methods that estimate values by averaging total rewards observed over complete episodes.
  • Motion Tracking Policy Humanoid controller trained in simulation to follow recorded human motions, such as dances, while staying balanced.
  • Multi-Agent Reinforcement Learning MARL Reinforcement learning with several agents learning together in a shared environment, cooperating, competing or both.
  • Multi-Task Learning Training one model on many tasks at once so learning for one helps the others.
  • Multilayer Perceptron MLP Simplest deep neural network, with fully connected layers, often used as a fast walking controller for legged robots.
  • Multimodal Action Distribution Case where several quite different actions are all correct at the same moment, which naive averaging models get wrong.
  • Neural Network Computer model of layered connected units whose connection strengths are adjusted in training to map inputs to outputs.
  • Observation History Short window of recent sensor readings or frames given to a robot controller as short-term memory.
  • Observation Space Everything a robot controller receives as input, such as camera images, joint positions, forces and instructions.
  • Off-Policy Learning Reinforcement learning that can learn from experience gathered by any source, including old versions of itself or humans.
  • Offline Reinforcement Learning Offline RL Improving robot behavior using only a fixed set of previously recorded experience, with no further practice.
  • On-Policy Learning Reinforcement learning that updates only with fresh experience from the current behavior, discarding older data.
  • Online Reinforcement Learning Reinforcement learning in which the agent keeps acting and learning from fresh experience rather than a fixed recording.
  • Out-of-Distribution OOD Describes inputs outside the range of situations a model saw in training, such as unfamiliar objects or lighting.
  • Overfitting When a model memorizes training examples too closely, including irrelevant details, and performs poorly on anything new.
  • Partially Observable Markov Decision Process POMDP Reinforcement learning problem in which the agent cannot see the full situation and must act on incomplete information.
  • Perceiver-Actor PerAct Instruction-following manipulation model that voxelizes a 3D workspace view and predicts the next gripper pose.
  • Policy Decision-making function that turns what a robot senses into what it does next, from simple rules to large AI models.
  • Policy Distillation Compressing one or more trained robot controllers into a single, often smaller, controller trained to copy them.
  • Policy Evaluation Measuring how well a trained robot controller performs in simulation, on hardware or in a learned world model.
  • Policy Gradient Reinforcement learning methods that directly adjust a policy toward behavior that earned higher reward in trial runs.
  • Privileged Information Data available during simulated training but not on the real robot, such as exact friction or object positions.
  • Proximal Policy Optimization PPO Most widely used reinforcement learning algorithm for simulated robot training, improving the policy in small clipped steps.
  • Q-Function Reinforcement learning estimate of total future reward from taking a given action in a given situation.
  • Q-Learning Classic off-policy reinforcement learning method that updates action values toward the best estimated next action value.
  • Rapid Motor Adaptation RMA Method that lets a walking robot infer terrain or load changes from recent joint readings and adjust its gait instantly.
  • Real-World Reinforcement Learning A physical robot learning by trial and error in the real world rather than in a simulator.
  • Recurrent Neural Network RNN Neural network that processes sequences step by step while carrying a running memory forward.
  • REINFORCE Simplest policy gradient algorithm, making actions that led to high total reward more likely.
  • Reinforcement Learning RL Learning by trial and error, where an agent tries actions, receives rewards or penalties and adjusts its behavior.
  • Reinforcement Learning from Human Feedback RLHF Training an AI from human ratings or comparisons of its outputs, then optimizing it to produce more of what people preferred.
  • Replay Buffer Memory store of a learner's past experience that it can sample and study again many times.
  • Representation Learning Teaching a model to turn raw input such as pixels into a compact, meaningful summary that makes later decisions easier.
  • Residual Learning Learning a small correction added on top of an existing controller rather than learning the whole behavior.
  • Return Total reward an agent collects from a moment until the episode ends, usually with later rewards discounted.
  • Reward Function Scoring rule that tells a reinforcement learning agent how well it is doing at each step.
  • Reward Hacking A learning agent exploiting an unintended loophole to earn a high score without doing the intended task.
  • Reward Model Learned model that estimates how well a robot is doing or how close it is to finishing a task.
  • Reward Shaping Adding intermediate rewards that guide a learning agent toward a goal and speed up training.
  • Robot Learning Field that uses machine learning to give robots skills from data and experience instead of hand programming every motion.
  • Rollout Running a robot controller in a real or simulated environment for one or more attempts to gather data or test it.
  • Safe Reinforcement Learning Safe RL Reinforcement learning that must respect safety limits during training or deployment, such as force or zone constraints.
  • Sample Efficiency How much data or practice a learning method needs to reach a given level of skill.
  • SARSA On-policy reinforcement learning algorithm that learns action values from the actions its current behavior actually takes.
  • Self-Supervised Learning SSL Training on raw unlabeled data by solving puzzles built from the data itself, such as predicting hidden image parts.
  • Soft Actor-Critic SAC Off-policy maximum-entropy reinforcement learning algorithm for continuous control, popular for learning on real robots.
  • Sparse Reward Reward given only when a task is fully completed, easy to define but hard to learn from.
  • Success Detector Learned model that judges from images or sensor data whether a robot completed its task, replacing a human judge.
  • Success Rate Share of test attempts in which a robot completes its task, the most common headline number in robot learning.
  • Supervised Fine-Tuning SFT Further training of a pretrained model on labeled examples of the wanted behavior, such as task demonstrations.
  • Target Network Slowly updated copy of a network used to compute stable learning targets in deep reinforcement learning.
  • Task Progress Score Test score giving partial credit for each completed stage of a task rather than only full success or failure.
  • Teacher-Student Learning Two-stage training where a teacher learns with perfect simulator information and a student imitates it using real sensors.
  • Temporal Difference Learning TD Learning value estimates step by step by updating toward the reward received plus the estimated value of the next state.
  • Temporal Ensembling Smoothing trick that blends overlapping predicted action chunks for the current moment to avoid jerky handovers.
  • Training and Inference The learning phase of a model on data and the usage phase when the finished model makes decisions, often onboard.
  • Transfer Learning Reusing what a model learned on one task or data type to get a head start on another.
  • Trust Region Policy Optimization TRPO Policy gradient method that limits how much the policy changes per update for stability, a forerunner of PPO.
  • Twin Delayed Deep Deterministic Policy Gradient TD3 Continuous-control reinforcement learning algorithm that fixes DDPG's instability with twin critics and delayed updates.
  • U-Net U-shaped neural network that compresses then expands its input with skip links, used in image generators and some action generators.
  • Value Function Estimate of the total future reward an agent can expect starting from a given situation.
  • Value Iteration Classic planning algorithm that repeatedly applies the Bellman optimality equation until state values stop changing.
  • Variational Autoencoder VAE Neural network that compresses data into a smooth latent space and reconstructs it, so new examples can be sampled.
  • Visuomotor Policy Robot policy that maps camera images, usually with joint readings, directly to motor commands.
  • Zero-Shot Generalization Handling a task, object or place never seen in training without extra examples or retraining.

Full entry for members

Every term has a full entry for members: the plain-words definition, the practitioner note and the typed links between terms.

See membership

ACW Robotics and Physical AI Glossary, edition 2026.11.1. The headword and one line are open to everyone.