Embodied Scene Rearrangement Planning

Noosaga Placements

• Roboticssubfield90%
  The paper focuses on embodied agents performing rearrangement tasks in 3D scenes, which is a core topic in robotics and embodied AI.
  Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied agents to rearrange furniture in 3D scenesIndex Terms—Mobile Manipulation, Semantic Scene Understanding, Scene Rearrangement Planning, Embodied AI.
• Learning-Based Roboticsframework80%
  The paper presents learning-based baselines (IL and RL) and discusses their performance, situating the work within learning-based robotics.
  two learning-based approaches (IL and RL)The IL baseline utilizes Behavior Cloning (BC) [12]The RL baseline employs Proximal Policy Optimization (PPO) [11]
• Robot Learningsubfield80%
  The paper includes learning-based baselines (IL and RL) and discusses their performance, making robot learning a relevant subfield.
  two learning-based approaches (IL and RL)The IL baseline utilizes Behavior Cloning (BC) [12]The RL baseline employs Proximal Policy Optimization (PPO) [11]
• Learning from Demonstrationframework70%
  The imitation learning baseline (ESRP-BC) uses behavior cloning with a diffusion policy, which is a form of learning from demonstration.
  The IL baseline utilizes Behavior Cloning (BC) [12]ESRP-BC We deploy a conditional diffusion policy [13]
• Planning And Searchsubfield70%
  The paper introduces a planning-based baseline (ESRP-PLAN) that uses task and motion planning, and discusses long-horizon planning challenges.
  a hierarchical task-and-motion planning methodESRP-PLAN baseline implements a Task and Motion Planning (TAMP) framework
• Reinforcement Learningframework70%
  The reinforcement learning baseline (ESRP-PPO) uses PPO, a model-free RL algorithm, and the paper discusses RL challenges.
  The RL baseline employs Proximal Policy Optimization (PPO) [11]ESRP-PPO consistently outperforms ESRP-BC across all three metrics
• Computer Visionsubfield60%
  The task involves egocentric RGB observations and top-down layout images, requiring visual scene understanding and alignment.
  egocentric RGB image I_ego (128 × 128)top-down RGB image I_g (128 × 128) depicting the target furniture layout
• Markov Decision Processesframework60%
  The task is formulated as a sequential decision-making problem with discrete actions, which can be modeled as a Markov Decision Process.
  We define the action space A = {move_forward, move_backward, turn_left, turn_right, fetch, release}The RL baseline employs Proximal Policy Optimization (PPO) [11]
• Reinforcement Learningsubfield60%
  One of the baselines is a reinforcement learning agent (ESRP-PPO) trained with PPO, and the paper discusses RL challenges.
  The RL baseline employs Proximal Policy Optimization (PPO) [11]ESRP-PPO consistently outperforms ESRP-BC across all three metrics
• Plan-Space Planningframework50%
  The planning-based baseline (ESRP-PLAN) uses task and motion planning, which involves task-level planning and motion planning.
  ESRP-PLAN baseline implements a Task and Motion Planning (TAMP) frameworkThe task planner determines the rearrangement order, and for each object the motion planner runs a four-phase routine

Abstract

This paper introduces Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied agents to rearrange furniture in 3D scenes to match a target configuration using only egocentric observations and a top-down target layout. Unlike prior rearrangement tasks, ESRP precludes global state access and introduces mutual object occlusions, reflecting the practical constraints of real-world robotic deployment. These factors make aligning partial egocentric observations with the global target layout particularly challenging for long-horizon planning. To facilitate research, we present ESRP-Bench, a comprehensive benchmark built on OmniGibson featuring over 5,400 scene pairs and 8,200 objects. We define three multi-level metrics to evaluate rearrangement quality and provide four baselines: a hierarchical task-and-motion planning method, a vision-language-model-based method, and two learning-based approaches (IL and RL). Experimental results demonstrate that current methods struggle to complete the task efficiently, highlighting ESRP as a challenging frontier for embodied agents in scene understanding and long-horizon task planning. This work serves as a stepping stone toward deploying intelligent agents in real-world scenarios. Project page: https://pie-lab.cn/ESRP/.

Paper Context