Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

Noosaga Placements

• Roboticssubfield95%
  The paper focuses on embodied question answering in robotics, evaluating memory architectures on a mobile robot in both simulation and real-world settings.
  Embodied question answering (EQA) is a core AI capability that requires a physical agent to perceive, reason, and interact within an environment to answer natural language queries grounded in the physical world.We further validate these findings on a real-world mobile robot, demonstrating that spatially grounded visual memory is critical for enabling continuous, intelligent operation in physical environments.
• Occupancy Grid Mappingframework80%
  The paper compares against ExploreEQA, which uses 2D occupancy maps as its memory representation, and shows its limitations for sequential EQA.
  ExploreEQA maintains a 2D occupancy map encoding traversable space and frontier scores.Agents that retain only traversability information, such as 2D occupancy maps, remember where the robot has explored but not the visual-semantic evidence needed for later questions.
• Robot Perception And Slamsubfield80%
  The paper extensively discusses memory architectures that involve mapping, spatial representations, and 3D geometry, which are core to robot perception and SLAM.
  architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation.3D-Mem constructs a persistent metric reconstruction of the environment, binding visual embeddings directly to 3D coordinates.
• Semantic SLAMframework70%
  The paper evaluates memory architectures that build spatial-semantic representations, which are closely related to Semantic SLAM approaches.
  3D-Mem constructs a persistent metric reconstruction of the environment, binding visual embeddings directly to 3D coordinates.architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation.
• Mobile Roboticssubfield70%
  The paper evaluates agents on a mobile robot (Unitree Go2) performing navigation and question answering in indoor and outdoor environments.
  We deploy a Unitree Go2 quadruped with an Intel RealSense D435i depth camera and onboard LiDAR L2.The environments tested are a furnished indoor lab space, an open lobby, and a long hallway.
• Deep Reinforcement Learningframework50%
  The paper evaluates UniNavid, a VLA model that is trained end-to-end, which may involve reinforcement learning, but the paper does not focus on RL methods.
  UniNavid finetunes a transformer-based VLM to directly predict navigation waypoints from a short window of sequential RGB frames and a language query.

Abstract

Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. However, real-world robots operate continuously and must accumulate, retain, and selectively reuse information acquired from prior interactions. Despite this practical requirement, the architectural mechanisms needed to support sequential memory in EQA remain underexplored. In this work, we investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries. We find that simply preserving existing memory is often insufficient. Agents that retain only traversability information, such as 2D occupancy maps, remember where the robot has explored but not the visual-semantic evidence needed for later questions. Agents trained on short-horizon episodic data face a different challenge: when exposed to continuous, multi-query histories, their inherited context suffers from severe temporal mismatch, rather than forming a reusable scene representation. To overcome this architectural bottleneck, we highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation. Extensive experiments in simulated environments reveal that this form of memory breaks the accuracy-efficiency tradeoff in sequential settings, simultaneously achieving higher answer accuracy and lower navigation costs. We further validate these findings on a real-world mobile robot, demonstrating that spatially grounded visual memory is critical for enabling continuous, intelligent operation in physical environments.

Paper Context