$τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation

Noosaga Placements

• Robot Manipulationsubfield95%
  The paper focuses on long-horizon robot manipulation, with tasks involving object manipulation, bimanual coordination, and mobile manipulation.
  Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks.The evaluation covers room cleaning, meal preparation, tea making, and laundry collection, with episodes lasting up to 12 minutes.
• Learning-Based Roboticsframework90%
  The paper is situated in the learning-based robotics framework, as it uses learned policies and models for robot control.
  The policy is trained on 40,115 hours of heterogeneous real-world data with multimodal co-training.The low-level policy couples a vision-language backbone with a Mixture-of-Transformers action expert.
• Robot Learningsubfield90%
  The paper presents a robot foundation model trained on large-scale heterogeneous data, with a focus on learning-based policies for manipulation.
  The policy is trained on 40,115 hours of heterogeneous real-world data with multimodal co-training.The low-level policy couples a vision-language backbone with a Mixture-of-Transformers action expert.
• Data-Driven and Learning-Based Manipulationframework85%
  The paper focuses on data-driven and learning-based manipulation, using large-scale data to train manipulation policies.
  The policy is trained on 40,115 hours of heterogeneous real-world data.The low-level policy is a generalist VLA trained on diverse manipulation data.
• Planning And Searchsubfield80%
  The high-level policy uses beam search over candidate subtasks, which is a search-based planning method.
  The high-level policy uses world-model-guided test-time computation to search over subtask sequences.Beam search retains promising branches, after which a reflective model commits to the next subtask.
• Self-Supervised Learning from Physical Interactionframework70%
  The paper uses self-supervised learning from physical interaction, as the high-level policy is trained with automatically generated supervision from demonstrations and perturbations.
  Supervision for the high-level policy is derived automatically from existing task, stage, and executable-subtask annotations.We additionally construct memory-perturbed examples that teach the policy to recover when its execution history lags behind.
• Deep Learningsubfield70%
  The system uses deep learning models including vision-language models, world models, and value networks.
  The proposal, value, and reflective models are independently fine-tuned from the same robot-pretrained VLM checkpoint initialized from Qwen3.5-9B.The world model is initialized from Step1X-Edit.
• Task-Oriented and Semantic Manipulationframework60%
  The paper addresses task-oriented and semantic manipulation, as the high-level policy generates subtasks based on language instructions and semantic understanding.
  The high-level policy generates subtasks conditioned on the task instruction and execution memory.The low-level policy executes language-conditioned actions.
• Roboticssubfield60%
  The paper is about a robot foundation model, which falls under the broader AI robotics category.
  We introduce τ0-VLA, a hierarchical robot foundation model.
• Hierarchical Task Network Planningframework50%
  The high-level policy uses hierarchical task network planning implicitly through subtask generation, though not explicitly as HTN.
  The high-level policy generates subtasks and sequences them coherently over extended tasks.

Abstract

Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks. Most hierarchical vision-language-action (VLA) models make each such decision with a single forward pass, leaving no mechanism to allocate additional computation to difficult or consequential choices. We introduce $τ_0$-VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem through world-model-guided test-time computation. At each inference step, the high-level policy uses execution memory to generate a subtask and, when needed, searches over alternatives before committing to its output. A low-level policy then executes the generated subtask across multiple robot embodiments. The policy is trained on 40,115 hours of heterogeneous real-world data with multimodal co-training. Across in-domain and distribution-shifted settings, allocating additional test-time computation substantially improves next-subtask prediction accuracy, and these gains translate into higher closed-loop success on long-horizon robot manipulation tasks.

Paper Context