Controllable Sim Agents with Behavior Latents

Noosaga Placements

• Robot Learningsubfield90%
  The paper proposes a learning-based framework for controllable traffic simulation, focusing on imitation learning and generative modeling of agent behavior, which is core to robot learning.
  We introduce Controllable Neural Variational Agents (CNeVA), a controllable simulated-agent framework that learns to infer a per-agent Gaussian behavior latent from per-channel discounted returns via a closed-form conjugate variational update, conditioning a rectified-flow trajectory generator trained on a mixed channel-mask curriculum for classifier-free guidance.CNeVA instead infers a closed-form posterior behavioral latent over an explicit reward basis, which acts as a single guidance token for a rectified-flow generator, improving the realism and diversity of controllable generation.
• Imitation Learningframework90%
  The paper uses imitation learning as a baseline and builds upon it, but the core method is a generative model that infers behavior latents, which is a form of imitation learning from offline data.
  Although imitation learning can enable the generation of diverse and realistic traffic behaviors (Seff et al., 2023; Wu et al., 2024; Philion et al., 2024; Zhang et al., 2025a), it generally lacks fine-grained controllability over agent behavior and therefore struggles to adapt generated scenarios to user-specified objectives.
• Diffusion Modelsframework80%
  The paper uses a rectified-flow trajectory generator, which is a type of diffusion model, and employs classifier-free guidance, a technique from diffusion models.
  We optimize this objective via conditional flow matching (Lipman et al., 2023), a simulation-free surrogate that regresses a neural velocity field onto the rectified-flow target instead of evaluating the likelihood directly.Inference uses classifier-free guidance (Ho and Salimans, 2022).
• Reinforcement Learningsubfield70%
  The framework uses discounted returns and reward channels, and the paper discusses reward hacking and eligibility gates, which are concepts from reinforcement learning, though the method is primarily offline and does not involve online RL training.
  To tackle scarcity in reward signals, we propose soft eligibility gates that replace hard binary thresholds with smooth exponential decay, preserving the gradient signal for near-threshold agents.Self-play reinforcement learning encourages desired behaviors through hand-crafted reward functions in closed-loop interaction... Despite their controllability, these methods rely on computationally expensive online training and typically require retraining whenever the reward specification changes.
• Deep Learningsubfield60%
  The method uses a rectified-flow trajectory generator, a type of deep generative model, and employs transformer architectures and classifier-free guidance, which are deep learning techniques.
  We optimize this objective via conditional flow matching (Lipman et al., 2023), a simulation-free surrogate that regresses a neural velocity field onto the rectified-flow target instead of evaluating the likelihood directly.Architecturally, λn enters as an extra cross-attention token prepended to the per-agent scenario-context conditioning set.
• Actor-Critic Methodsframework50%
  The paper discusses reward channels and discounted returns, which are concepts from reinforcement learning, but the method does not use actor-critic methods; it is situated in the context of RL for controllability.
  Self-play reinforcement learning encourages desired behaviors through hand-crafted reward functions in closed-loop interaction... Despite their controllability, these methods rely on computationally expensive online training and typically require retraining whenever the reward specification changes.
• Roboticssubfield50%
  The paper addresses traffic simulation for autonomous driving, which is a robotics application, but the focus is on the learning and control framework rather than physical robotic systems.
  Realistic traffic simulation requires agents that imitate logged behavior and can also be steered along interpretable axes. Such controllability enables engineers to isolate variables, reproduce specific edge cases, and test autonomous systems without real-world risk.
• Safe Robot Learningframework40%
  The paper discusses safety controllability and reward hacking, which are relevant to safe robot learning, but the framework is not primarily about safety guarantees.
  Safety controllability is monotone and substantial with the introduction of soft eligibility.Furthermore, our experiment demonstrates that steering metrics must be read alongside physical-plausibility guardrails to avoid reward-hacking confounds.

Abstract

Realistic traffic simulation requires agents that imitate logged behavior and can also be steered along interpretable axes. Such controllability enables engineers to isolate variables, reproduce specific edge cases, and test autonomous systems without real-world risk. We introduce Controllable Neural Variational Agents (CNeVA), a controllable simulated-agent framework that learns to infer a per-agent Gaussian behavior latent from per-channel discounted returns via a closed-form conjugate variational update, conditioning a rectified-flow trajectory generator trained on a mixed channel-mask curriculum for classifier-free guidance. To tackle scarcity in reward signals, we propose soft eligibility gates that replace hard binary thresholds with smooth exponential decay, preserving the gradient signal for near-threshold agents. On the Waymo Open Motion Dataset, CNeVA attains competitive realism on the benchmark while exposing per-channel controllability that the higher-ranked imitation models lack. Speed- and acceleration-based steering produces monotone responses without stall-induced reward hacking. Safety controllability is monotone and substantial with the introduction of soft eligibility. We manage to achieve steerable map compliance under a context-residual return measure. Furthermore, our experiment demonstrates that steering metrics must be read alongside physical-plausibility guardrails to avoid reward-hacking confounds.

Paper Context