Loading field map
Robot learning is the subfield of robotics concerned with enabling robots to acquire skills and behaviors through data and experience rather than through purely manual programming. Its central question is deceptively simple: how can a physical system—with a body, sensors, and actuators—learn to act competently in a world that is only partially known and constantly changing? The stakes are practical as well as scientific. Robots that must operate in homes, hospitals, warehouses, or disaster sites face situations their designers cannot fully anticipate. If a robot can learn, it can adapt to new objects, new layouts, and new tasks without a human rewriting its software for every contingency.
The field sits at the intersection of robotics, machine learning, and control theory, but it is not merely the application of machine learning to robots. Robot learning must contend with the fact that the robot's actions change the world it perceives, that data collection is slow and physically costly, that mistakes can damage hardware or people, and that the robot's body imposes real constraints on what can be learned. These constraints shape every major approach in the field.
To understand robot learning, it helps to distinguish three related but distinct problems. The first is perception: the robot must infer the state of the world from noisy, partial sensor data. The second is policy learning: the robot must decide what action to take given its current state and goal. The third is skill acquisition: the robot must improve its ability to perform a task over repeated attempts. These problems are deeply intertwined. A robot cannot learn a good policy if it cannot perceive the relevant features of its environment, and it cannot improve its perception without acting to gather informative data.
A further distinction cuts across all three: the difference between model-based and model-free approaches. A model-based method builds an explicit representation of the world's dynamics—how the state changes in response to actions—and uses that model to plan or to simulate experience. A model-free method skips the explicit model and learns a policy or value function directly from interaction data. This is not a rivalry between two schools so much as a fundamental design choice with trade-offs. Model-based methods can be sample-efficient because they exploit structure, but they suffer when the model is wrong. Model-free methods are more flexible but typically require far more data, which is expensive for physical robots.
Robot learning emerged as a recognizable field in the late 1980s and early 1990s, but its intellectual roots go back further. In the 1960s and 1970s, researchers in artificial intelligence explored robots that could plan their actions using symbolic representations of the world. These systems, such as Shakey the robot at Stanford Research Institute, operated by reasoning over logical descriptions of objects and actions. They were not learners in the modern sense; they were planners that assumed the world could be described accurately in advance. The connection to robot learning is indirect: these systems established the idea that robots could be more than pre-programmed machines, but their brittle dependence on hand-coded world models eventually motivated a turn toward learning.
A more direct precursor came from reinforcement learning (RL), a branch of machine learning in which an agent learns to choose actions by maximizing a scalar reward signal. The formal foundations of RL were laid in the 1950s and 1960s with work on dynamic programming and optimal control, and the term itself was consolidated in the 1980s through the work of Richard Sutton and Andrew Barto. By the early 1990s, researchers began applying RL to simple robot tasks, such as learning to balance a pole or to navigate a small maze. These early demonstrations were important proof-of-concept results, but they also revealed the central difficulty of robot learning: tabula rasa RL requires thousands or millions of trials, while a physical robot can only perform a few hundred attempts before wearing out its motors or running out of patience.
The 1990s also saw the rise of learning from demonstration (LfD), sometimes called imitation learning. The idea was intuitive: instead of letting a robot discover a skill through trial and error, a human teacher demonstrates the skill, and the robot learns to reproduce it. This approach drew on earlier work in teleoperation and on the broader AI interest in learning by example. LfD proved attractive because it sidesteps the sample-efficiency problem of RL. A human can demonstrate a task a handful of times, and the robot can extract a policy from those demonstrations. The difficulty lies in a problem known as covariate shift: a policy trained on demonstrations may perform well when the robot stays near the demonstrated states, but it can drift into unfamiliar states where it has no guidance, and errors compound. This problem motivated a family of algorithms that interleave imitation with corrective feedback or with additional RL fine-tuning.
A third major strand developed through the 2000s: learning for manipulation. As robotic hardware improved and as the research community focused on tasks like grasping and assembly, it became clear that traditional analytical approaches—computing a grasp from a geometric model of the object—were too brittle. Objects vary in shape, compliance, friction, and surface texture, and these properties are hard to model accurately. Researchers began to treat grasping as a learning problem: collect data on successful and failed grasps, and train a model to predict which grasps will work. This line of work connected robot learning to the broader deep learning revolution of the 2010s, when neural networks trained on large datasets transformed computer vision and, subsequently, robot perception.
The modern field is organized less by a single dominant paradigm than by a set of approaches that address different parts of the problem space. These approaches overlap, combine, and borrow from one another, but each has a distinct core idea.
Reinforcement learning remains the most ambitious and most general framework for robot learning. In RL, the robot is an agent that interacts with an environment over a sequence of time steps. At each step, it observes a state, chooses an action, receives a reward, and transitions to a new state. The goal is to learn a policy—a mapping from states to actions—that maximizes cumulative reward. The appeal is that RL requires no prior knowledge of the task beyond the reward signal. The robot can, in principle, discover strategies that a human designer would never have considered.
The central obstacle is sample efficiency. A robot learning to grasp an object from scratch might need millions of trials, each of which takes seconds of real time and risks hardware damage. The field has responded with several strategies. Simulation-to-real transfer trains the policy in a physics simulator, where trials are cheap and parallel, and then attempts to transfer the learned policy to the physical robot. This works when the simulator is accurate enough, but the gap between simulation and reality—known as the sim-to-real gap—is a persistent source of failure. Researchers have developed techniques like domain randomization, which varies the simulator's parameters randomly during training so that the learned policy becomes robust to the discrepancies it will encounter in the real world. Another strategy is off-policy learning, which allows the robot to reuse past experience rather than discarding it after each update, improving data efficiency. A third is model-based RL, which learns a dynamics model from experience and then uses that model to plan or to generate additional training data.
RL has achieved notable successes in robot locomotion, where simulated agents learn to walk and run, and in manipulation, where policies learned in simulation can grasp novel objects. But the field remains constrained by the difficulty of specifying reward functions that capture what the user actually wants. A robot rewarded for grasping an object quickly may learn to throw it rather than place it gently. This problem, sometimes called reward hacking, is a reminder that RL optimizes the reward, not the intent behind it.
Learning from demonstration addresses the sample-efficiency problem by starting from human knowledge. The robot is given a set of demonstrations—typically kinesthetic (a human physically guides the robot's arm), teleoperated (a human controls the robot remotely), or observational (a human performs the task while the robot watches). The robot must infer the underlying skill.
The simplest approach is behavioral cloning: treat the demonstrations as supervised learning data, where the inputs are states and the outputs are actions, and train a policy to mimic them. Behavioral cloning is straightforward but suffers from covariate shift. Because the policy is trained on states visited by the demonstrator, it never learns to recover from its own mistakes. If the robot deviates slightly from the demonstrated trajectory, it enters a state it has never seen, and its predictions become arbitrary.
More sophisticated approaches address this limitation. Interactive imitation learning algorithms, such as DAgger (Dataset Aggregation), have the robot execute its current policy, ask a human to correct its actions in the states it actually visits, and add those corrections to the training set. This iteratively closes the gap between the training distribution and the deployment distribution. Another family of methods, inverse reinforcement learning (IRL), does not try to imitate actions directly. Instead, it infers the reward function that the demonstrator was implicitly optimizing, and then uses RL to find a policy that optimizes that inferred reward. IRL is powerful because it can generalize beyond the demonstrations, but it is computationally expensive and can infer the wrong reward if the demonstrations are noisy or suboptimal.
Learning from demonstration is widely used in industry for tasks like assembly and pick-and-place, where a human can quickly show a robot what to do. Its main limitation is that it inherits the biases and limitations of the demonstrator. A robot trained on demonstrations cannot exceed the quality of its teacher, and it may struggle with tasks that require exploration or novel strategies.
A third approach, sometimes called learning by trial and error or self-supervised learning, sits between RL and LfD. Instead of starting from demonstrations or from a hand-designed reward, the robot learns from its own physical interactions with the world, using simple, task-agnostic feedback signals. The canonical example is grasping: a robot attempts to grasp an object, and the success signal is simply whether its gripper closes around the object and lifts it. This binary feedback is easy to obtain automatically, without human labeling, and the robot can collect thousands of attempts by repeatedly trying to grasp objects in a bin.
This approach was pioneered in the mid-2010s by researchers who showed that a robot could learn to grasp novel objects by training a convolutional neural network on a large dataset of its own grasp attempts. The key insight was that the robot itself generates the training data, so the data is always relevant to the robot's own embodiment and sensor suite. The approach scales with robot time: more attempts yield better performance. Its limitations are equally clear. The feedback signal must be automatically computable, which restricts it to tasks with clear success criteria. And the robot's exploration is limited by its own prior experience; it may never discover strategies that require unusual or risky actions.
A fourth approach emphasizes learning an explicit model of the world and using it for planning. The robot learns a predictive model: given the current state and an action, what will the next state be? Once the model is learned, the robot can use planning algorithms—such as model predictive control (MPC)—to search for action sequences that achieve a goal. The advantage is sample efficiency: the model can be learned from relatively few interactions, and the planner can evaluate many hypothetical futures without touching the physical robot.
The challenge is that learned models are imperfect, and errors compound over long horizons. A model that predicts the next state with 95% accuracy may be useless for planning a 100-step sequence, because the errors accumulate. Modern approaches address this by planning in a model predictive control loop: plan a short horizon, execute the first action, observe the actual outcome, and re-plan. This closed-loop strategy limits the damage from model error. Another strategy is to learn a model of the effects of actions that matter for the task, rather than a full model of the world. For example, a robot learning to push objects might learn only how objects move in response to pushes, not the full physics of the scene.
Model-based learning is particularly important in manipulation, where the robot must reason about contact, friction, and object dynamics. It also connects robot learning to classical control theory, which has long used models for planning and feedback. The relationship between model-based and model-free methods is not a rivalry but a spectrum. Many modern systems combine both: learn a model to generate simulated experience, then use model-free RL to optimize a policy on that experience.
No account of robot learning is complete without addressing the central role of simulation. Physical robots are slow, expensive, and fragile. A single grasp attempt might take five seconds, and a robot can run for only a few hours before needing maintenance. A simulator, by contrast, can run thousands of trials per second on a single GPU. This disparity has made simulation the primary training ground for many robot learning systems.
The cost is the sim-to-real gap. Simulators approximate physics, and their approximations are wrong in ways that matter. Friction coefficients are uncertain, contact dynamics are simplified, and sensor noise is modeled imperfectly. A policy trained in simulation may exploit simulator artifacts—such as unrealistic friction or perfect sensing—that do not exist in the real world. The field has developed several responses. Domain randomization trains the policy across a distribution of simulated environments with randomized parameters, so that the policy learns behaviors that work across many possible worlds, including the real one. System identification attempts to calibrate the simulator to match the real robot as closely as possible. Reality gap minimization uses real-world data to fine-tune the policy after simulation training.
The current consensus is that simulation is indispensable but insufficient. The most reliable systems use simulation for initial training and real-world data for fine-tuning or validation. The balance between simulation and real-world experience remains one of the field's most active design questions.
The field today is characterized by a convergence of deep learning, large datasets, and increasingly capable hardware. Deep neural networks have become the default function approximators for policies, value functions, and dynamics models, because they can represent the complex, high-dimensional mappings that robot tasks require. The availability of large-scale compute has made it possible to train these networks on millions of interactions, at least in simulation.
Several trends define the current landscape. Foundation models—large models pre-trained on diverse data—are beginning to enter robot learning. Vision-language models can provide robots with general knowledge about objects and tasks, and some systems use them to generate reward functions or to interpret natural language instructions. This is a young and rapidly evolving area, and its long-term impact is not yet clear. Multi-task learning aims to train a single policy that can perform many tasks, rather than one policy per task. This is motivated by the observation that skills share structure: grasping a cup and grasping a bottle involve similar motor primitives. Offline reinforcement learning seeks to learn policies from fixed datasets of experience, without any further interaction with the environment. This is attractive because it would allow robots to learn from data collected by other robots or by humans, but it is technically difficult because the learner cannot explore to correct its mistakes.
The field's open problems are substantial. Sample efficiency remains the fundamental bottleneck: even with simulation, learning complex manipulation skills requires enormous amounts of data. Generalization is unsolved: a policy trained on one set of objects or environments often fails on slightly different ones. Safety is a growing concern, as learning systems are deployed on physical hardware that can injure people or damage property. Long-horizon tasks—tasks that require hundreds or thousands of steps, like assembling a piece of furniture—remain beyond the reach of current methods, which struggle to maintain coherence over long sequences. And reward specification continues to be a practical headache: designing a reward function that captures the user's true intent is often harder than learning the policy itself.
Robot learning is not a settled field with a single accepted framework. It is a collection of approaches—reinforcement learning, learning from demonstration, self-supervised interaction, model-based planning—that address different aspects of the same underlying problem: how a physical agent can improve through experience. The approaches are complementary more than competitive. A modern robot learning system might use demonstrations to initialize a policy, simulation to refine it, real-world interaction to correct for the sim-to-real gap, and a learned model to plan when the policy is uncertain. The field's progress has been driven less by any single breakthrough than by the gradual accumulation of techniques that make learning from physical experience more reliable, more sample-efficient, and more general.