This paper introduces ODARE, a framework for online discovery and refinement of state and temporal abstractions in non-stationary environments. It integrates online change-point detection with hierarchical skill discovery, enabling agents to detect environmental shifts, discover new abstractions, and refine existing ones.
Key findings
ODARE addresses the abstraction gap in non-stationary RL by modeling the abstraction-environment interplay.
The framework provides a formal problem formulation, detailed algorithmic design, and theoretical analysis of regret bounds.
ODARE enables sample-efficient lifelong learning across diverse non-stationary scenarios.
Limitations & open questions
The paper is a research proposal and does not yet include experimental results.
The effectiveness of ODARE in real-world applications remains to be validated.