LIME: Learning Intent-aware Camera Motion from Egocentric Video

Noosaga Placements

• Robot Perception And Slamsubfield90%
  The paper directly addresses active perception and camera motion for acquiring visual evidence, which is core to robot perception. The method predicts relative camera poses to obtain better observations, a key aspect of active perception and SLAM.
  We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation.LIME learns intent-conditioned camera motion from passive egocentric video, turning ordinary human recordings into supervision for a reusable active-perception primitive.
• Active Visionframework90%
  The paper explicitly situates its work within the active perception literature, which is the core of this framework. The goal is to move the camera to acquire better visual evidence, a classic active perception problem.
  Active perception has long studied sensor motion for better observations [2, 3, 4].LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.
• Computer Visionsubfield90%
  The method heavily relies on computer vision techniques: processing RGB images, using a VLM backbone, predicting camera poses, and evaluating on visual benchmarks. The core task is vision-based.
  We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head.Given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation.
• Deep Imitation Learningframework80%
  The paper trains LIME from egocentric video data, which is a form of imitation learning from human demonstrations. The model learns to mimic the camera motions observed in human video, which is a core idea in imitation learning.
  we mine multi-intent camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses.LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.
• Robot Learningsubfield80%
  The paper trains a model (LIME) from data, which is a robot learning approach. It uses a VLM backbone and flow-matching head, and the learned policy is transferred to real robots.
  We propose LIME, a vision-language camera-motion generator trained from egocentric video frame pairs with mined intents, observation-gain descriptions, and relative camera poses.We further deploy our method on a Boston Dynamics Spot with an arm, using RGB-D images from its hand camera.
• Vision Transformers and Foundation Modelsframework70%
  The method uses a VLM (Qwen3-VL-4B) as its backbone, which is a vision-language foundation model. This framework covers the use of such models for vision tasks.
  We instantiate Qwen3-VL-4B-Instruct, freeze the vision encoder, and train the multimodal projector, language model, and flow-matching head.
• Deep Learningsubfield70%
  The method uses a VLM (Qwen3-VL-4B) and a flow-matching head, which are deep learning techniques. The training involves large-scale data and transformer architectures.
  We instantiate Qwen3-VL-4B-Instruct, freeze the vision encoder, and train the multimodal projector, language model, and flow-matching head.The continuous flow-matching head then models the conditional distribution of relative SE(3) target poses from this fused representation.
• Diffusion Modelsframework60%
  The paper uses a flow-matching head to model the distribution of target poses. Flow matching is a generative modeling technique related to diffusion models.
  We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head.The continuous flow-matching head then models the conditional distribution of relative SE(3) target poses from this fused representation.
• Natural Language Processingsubfield60%
  The method uses natural language as input (intents) and generates observation-gain descriptions. It leverages a VLM, which is a language model extended to vision.
  given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation.The language interface uses the VLM's autoregressive decoder to generate the observation-gain description.

Abstract

Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, language-conditioned camera motion remains comparatively underexplored as a first-class action. We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. This task is inherently non-trivial: viewpoint changes are driven by latent perceptual intentions, and a valid motion may operate at different semantic granularity, from entering a room to looking around a corner, inspecting a visible object, or revealing an occluded detail. To model this structure, we mine multi-intention camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses. We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across experiments and downstream robotic tasks, we show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.

Paper Context