From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model

Noosaga Placements

• Roboticssubfield95%
  The paper presents a new VLA model for robot manipulation, directly addressing viewpoint robustness in robot deployment. The method is evaluated on real and simulated robot tasks.
  We introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometryEvaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints.
• Robot Learningframework90%
  The paper is situated within the Robot Learning framework, as it proposes a learning-based VLA model that learns to predict actions and hand-eye matrices from data.
  CamVLA achieves viewpoint robustness by decoupling the policy into two parallel componentsWe optimize the model end-to-end with a joint objective
• Imitation Learningframework85%
  The paper uses imitation learning from expert demonstrations to train the VLA model, which is a form of imitation learning.
  For each task and viewpoint, we collect 100 expert demonstrations for training and 50 episodes for evaluation.We collect training demonstration data from five different camera perspectives
• Computer Visionsubfield80%
  The paper heavily relies on computer vision techniques: monocular RGB input, visual feature extraction, and geometric reasoning about camera poses. The primary arXiv category is cs.CV.
  requiring only a single monocular RGB image as the visual observation and task instruction at deploymentFrom the extracted visual representations, two specialized heads predict the camera-centric action and the hand-eye matrix in parallel
• Vision Transformers and Foundation Modelsframework80%
  The paper builds upon Vision Transformers and Foundation Models, specifically using VLM backbones (e.g., π0, GR00T N1.7) that are based on transformer architectures.
  We instantiate CamVLA upon foundational VLA architectures, specifically π0 and GR00T N1.7The auxiliary Geometric Head operates on the high-level semantic features extracted from the visual tokens of the backbone
• Robot Learningsubfield75%
  The paper proposes a learning-based approach for robot manipulation, specifically focusing on learning to predict actions and hand-eye matrices from data.
  CamVLA achieves viewpoint robustness by decoupling the policy into two parallel componentsWe optimize the model end-to-end with a joint objective
• Transformer Architectureframework75%
  The underlying VLA models (π0, GR00T N1.7) use Transformer architectures, and the paper's method is built on top of them.
  We instantiate CamVLA upon foundational VLA architectures, specifically π0 and GR00T N1.7, adhering to their respective training recipes and configurations.
• Deep Learningsubfield70%
  The model uses a VLM backbone and transformer-based architectures (π0, GR00T N1.7) with a lightweight MLP head, trained end-to-end with a joint objective.
  CamVLA is trained in a multi-task manner on 8 NVIDIA H100 80GB GPUsThe auxiliary Geometric Head is implemented as a lightweight three-layer Multi-Layer Perceptron (MLP) with GELU activations and a hidden dimension of 1024.
• Robot Manipulationsubfield70%
  The method is evaluated on manipulation tasks such as pick-and-place, pushing, and wiping, and the action space is end-effector delta poses.
  We evaluate CamVLA on six representative manipulation tasks: slide block to target, push buttons, take umbrella out of umbrella stand, close laptop lid, lamp off, and put knife on chopping board.The predicted robot actions are parameterized as delta 6-DoF end-effector poses relative to the current state
• Representation Learningframework70%
  The paper learns representations from visual and language inputs to predict actions and hand-eye matrices, which is a form of representation learning.
  From the extracted visual representations, two specialized heads predict the camera-centric action and the hand-eye matrix in parallel

Abstract

Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.

Paper Context