LT-Mem: Volatility-Aware Spatio-Temporal Memory for Lifelong Scene Understanding

Noosaga Placements

• Robot Perception And Slamsubfield90%
  The paper builds on multi-session SLAM for spatial alignment and integrates instance-level 3D perception, which are core to robot perception and SLAM.
  a multi-session SLAM backbone provides spatially aligned per-object observations across sessionsWe build on MASt3R-SLAM [10] for multi-session alignment and apply instance-level 3D segmentation [11]
• Graph-based SLAMframework90%
  The multi-session SLAM backbone uses factor graph optimization for alignment, which is a graph-based SLAM approach.
  The resulting factor graph, containing intra-session odometry edges and inter-session loop-closure edges, is optimized jointly using g2o
• Roboticssubfield85%
  The work addresses long-term robot operation and memory for scene understanding, which is a core robotics challenge.
  Long-term robot operation in evolving environments requires object-level understandingLT-Mem, a volatility-aware memory evolution framework
• Factor Graph Formulationframework80%
  The SLAM alignment is formulated as a factor graph with inter-session constraints, which is a factor graph formulation.
  The resulting factor graph, containing intra-session odometry edges and inter-session loop-closure edges
• Visual SLAMframework80%
  The perception layer uses MASt3R-SLAM, a visual SLAM system, for pose estimation and dense mapping.
  We build on MASt3R-SLAM [10], which estimates camera poses and dense point maps from monocular video
• Computer Visionsubfield70%
  The perception layer uses instance segmentation and 3D lifting from visual data, and the dataset includes visual QA, but the focus is on memory and reasoning.
  apply instance-level 3D segmentation [11] to extract per-object centroidsLT-VQA, a dataset and evaluation suite comprising multi-session recordings
• Geometric and Physical Reconstructionframework70%
  The perception layer performs 3D lifting of instance masks to obtain centroids and volumes, which is geometric reconstruction.
  each mask region is projected into the global frame to obtain per-instance 3D points
• Deep Learningsubfield50%
  The framework uses LLMs for reasoning and embeddings for retrieval, but the core contribution is not deep learning architecture.
  a constrained LLM used only for ambiguous casesobject-centric retrieval uses BGE-small-en-v1.5
• Statistical and Learning-Based Recognitionframework60%
  The framework uses instance segmentation and feature embeddings for object recognition and re-identification.
  apply instance-level 3D segmentation [11] to extract per-object centroids, volumes, and semantic embeddings
• Attention Mechanisms and Transformersframework50%
  The framework uses LLMs for reasoning and embeddings for retrieval, which are based on transformer architectures.
  a constrained LLM used only for ambiguous casesobject-centric retrieval uses BGE-small-en-v1.5

Abstract

Long-term robot operation in evolving environments requires object-level understanding that persists across repeated revisits. Existing systems either overwrite history to maintain an up-to-date map or store semantic snapshots without consistent cross-session object identity, resulting in temporal amnesia: the systematic loss of object history that prevents answering queries such as "Where has the green chair been across all sessions?" We propose LT-Mem, a volatility-aware memory evolution framework that unifies spatially aligned instance-level 3D perception with volatility-conditioned temporal reasoning. First, a multi-session SLAM backbone provides spatially aligned per-object observations across sessions. Second, a reasoning layer governs how object memory evolves: deterministic evidence scoring preserves cross-session identity, and a volatility-aware policy selects among overwrite, hold, and multi-hypothesis actions based on each object's dynamics. Third, the resulting Tri-Memory structure (Live, Delta, Meta) preserves both current states and event histories, enabling longitudinal object-centric reasoning. We further introduce LT-VQA, a dataset and evaluation suite comprising multi-session recordings, persistent identity annotations, and temporal QA pairs. Experiments show that LT-Mem consistently outperforms baselines across all metrics while consuming an order of magnitude fewer tokens, and ablations confirm that gains are driven by the structured memory architecture rather than LLM capacity.

Paper Context