Object-Oriented Latent Dynamics Graph
Vision-based RL agent with an object-centric world model for ATARI games.
Keywords: interpretable planning, world model, Graph Neural Networks, vision-based agent, VGDL, object-centric representation.
From October 2020 - August 2021, I researched under the supervision of Professor Michael Littman, I researched under the supervision of Professor Micahel Littman and collaborated with graduate student Mingxuan Li at Brown Human-Centered Robotics Initiative. Our work, Learning to Control via Object-Oriented Latent Dynamics Graph, proposed a novel vision-based and object-centric world model for RL agents. I proposed an unsupervised segmentation and clustering computer vision algorithm for extracting object-oriented game dynamics. We used PySR (a symbolic regression engine) to replace black-box Graph Neural Network components in our world model, so as to achieve with interpretable planning.
Unsupervised and object-centric segmentation and clustering:
