Mixture of Frames Policy: Multi-Frame Action Denoising for Bimanual Mobile Manipulation

Noosaga Placements

• Imitation Learningframework95%
  The paper uses imitation learning to train the diffusion policy from demonstrations. The method is a form of imitation learning for robot manipulation.
  We consider visuomotor imitation learning for bimanual mobile manipulation with diffusion policies.All policies are trained for 500 epochs with 100 demonstrations...
• Robot Manipulationsubfield95%
  The paper focuses on bimanual mobile manipulation, proposing a method for learning manipulation policies that reason across multiple coordinate frames. The method is evaluated on manipulation tasks such as pouring, serving, and assembly.
  We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames.Across nine simulated bimanual manipulation tasks...We further evaluate MoF on two real-world bimanual mobile manipulation tasks...
• Robot Learningsubfield95%
  The paper presents a learning-based method (diffusion policy with mixture of experts) for robot manipulation, trained via imitation learning. The core contribution is a new policy architecture for robot learning.
  We consider visuomotor imitation learning for bimanual mobile manipulation with diffusion policies.MoF augments the shared backbone with four frame experts and a learned router...All policies (MoF and all baselines) share the same diffusion backbone...
• Diffusion Modelsframework95%
  The paper builds on diffusion models, specifically the Diffusion Policy framework. The core of the method is a diffusion-based visuomotor policy.
  We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames.MoF can be implemented on top of a standard DDIM sampler...
• Learning-Based Manipulationframework90%
  The paper proposes a learning-based manipulation method. The MoF policy is a learned visuomotor policy for manipulation tasks.
  We propose Mixture of Frames Policy (MoF), a diffusion policy...Our real-world experiments study how multi-frame fusion improves robustness over singleframe policies in bimanual mobile manipulation.
• Roboticssubfield80%
  The paper addresses a fundamental challenge in robotic manipulation (frame selection) and proposes a general framework applicable to various robotic systems. It bridges AI methods (diffusion models, mixture of experts) with robotics.
  Robotic manipulation is inherently multi-frame...We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames.
• Deep Learningsubfield70%
  The method builds on diffusion models, a deep learning technique, and uses a mixture-of-experts architecture. The paper introduces a novel action representation and denoising procedure within the diffusion framework.
  MoF maintains a single canonical diffusion state...We introduce a column-based 6D rotation representation within an SE(3) action parameterization...MoF augments the shared backbone with four frame experts and a learned router...
• Deep Reinforcement Learningframework60%
  The paper compares against a Mixture-of-Experts Diffusion Policy (MoE-DP) baseline, which is a form of deep reinforcement learning or imitation learning with MoE. However, the paper itself does not use reinforcement learning.
  MoE-DP [29] uses a mixture-of-experts module in the conditioning pathway of DP, but still denoises actions in a single action representation.

Abstract

Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we introduce a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines. Project homepage: https://mofpo.github.io

Paper Context