This paper introduces HyperWorld, a model-based reinforcement learning framework that uses hyperbolic geometry for world model representations, capturing hierarchical temporal abstractions and compositional structure in environment dynamics.
Key findings
HyperWorld employs hyperbolic geometry to naturally capture hierarchical structures in sequential decision-making tasks.
The Hyperbolic Recurrent State-Space Model (RSSM-H) extends the standard RSSM to the Poincaré ball model of hyperbolic space.
Stable training procedures are developed to address optimization challenges specific to hyperbolic spaces in RL.
Theoretical analysis provides bounds on representation capacity and sample complexity, with practical solutions for gradient stability and policy optimization in hyperbolic latent spaces.
Limitations & open questions
The paper discusses practical challenges including gradient instability and trust region violations in hyperbolic RL.
Further work is needed to fully integrate hyperbolic geometry into world models and explore its implications for different RL tasks.