A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation

Noosaga Placements

• Robot Manipulationsubfield95%
  The paper focuses on dexterous manipulation, specifically contact-rich tool-use tasks like operating scissors and turning a screwdriver, which are core to robot manipulation.
  We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration.The resulting policies produce fluid, human-like behavior on two different multi-fingered hands across contact-rich tool-use tasks, including operating a pair of scissors and turning a screwdriver.
• Learning-Based Manipulationframework95%
  The paper uses learning-based manipulation, specifically reinforcement learning, to train dexterous manipulation policies.
  We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration.trains a residual RL policy in simulation to track object-centric keypoints along that reference
• Robot Learningsubfield90%
  The paper uses reinforcement learning to train policies in simulation, with a focus on learning from human demonstrations and sim-to-real transfer.
  We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration.trains a residual RL policy in simulation to track object-centric keypoints along that reference
• Deep Reinforcement Learningframework90%
  The paper uses deep reinforcement learning (PPO) with neural network policies for dexterous manipulation.
  We use PPO implemented in RSL-RL.Actor MLP hidden dims: 1024, 512, 256, 128
• Reinforcement Learningsubfield80%
  The paper uses reinforcement learning (PPO) as the core learning algorithm, with techniques like reference state initialization and residual actions.
  We use PPO implemented in RSL-RL.The policy πθ learns a residual action on top of the reference motion.
• Model-Free Reinforcement Learningframework80%
  The paper uses model-free reinforcement learning (PPO) to train the policy, without learning a dynamics model.
  We use PPO implemented in RSL-RL.The policy πθ learns a residual action on top of the reference motion.
• Robot Controlsubfield70%
  The paper involves low-level PD control and system identification for sim-to-real transfer, which are aspects of robot control.
  This control target is then sent to a low-level PD controller.We perform system identification to align the robot dynamics between simulation and the real-world system.
• Imitation Learningframework70%
  The paper uses a single human demonstration as a reference, which is a form of imitation learning, though the primary learning method is RL.
  We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration.The manipulation task is specified by a human demonstration τ̃

Abstract

Recent work in humanoid whole-body control has found success with a simple recipe: retarget human motion to robot kinematic references, then train policies via reinforcement learning (RL) to track them. But how does this recipe transfer to dexterous manipulation? The answer is not obvious, as manipulation involves complex, contact-rich dynamics and requires delicate regulation of contact modes and forces. We present REGRIND, a minimalist retargeting-guided RL pipeline that learns dexterous manipulation policies from a single human demonstration. REGRIND retargets human hand-object motion to a robot reference that preserves hand-object spatial and contact relationships, trains a residual RL policy in simulation to track object-centric keypoints along that reference, and transfers the resulting policy zero-shot to hardware with careful system identification. The resulting policies produce fluid, human-like behavior on two different multi-fingered hands across contact-rich tool-use tasks, including operating a pair of scissors and turning a screwdriver. Through systematic hardware experiments, we identify and analyze the key factors that govern sim-to-real transfer in dexterous manipulation, offering practical guidance for retargeting-based learning in contact-rich settings. Videos and code are available at https://yunhaifeng.com/REGRIND.

Paper Context