Where: https://cmu.zoom.us/j/92520469322?pwd=SjlpTVI5MGdtN1VBakFkRG82bStYQT09

Meeting ID: 925 2046 9322

Passcode: 323696

Calendar Invite: https://tinyurl.com/shuby-msr-thesis-talk-invite

Title: Towards Interpretable RL — Interactive Visualizations to Increase Insight

Abstract:

Visualization tools for supervised learning (SL) allow users to interpret, introspect, and gain an intuition for the successes and failures of their models. While reinforcement learning (RL) practitioners ask many of the same questions while debugging agent policies, existing tools aren’t a great fit for the RL setting as these tools address challenges typically found in the SL regime. Whereas SL involves a static dataset, RL often entails collecting new data in challenging environments with partial observability, stochasticity, and non-stationary data distributions. This necessitates the creation of alternate visual interfaces to help us better understand agent policies trained using RL. In this work, we design and implement an interactive visualization tool for debugging and interpreting RL. Our system identifies and addresses important aspects missing from existing tools and we provide an example workflow of how this system could be used, along with ideas for future extensions. We explain one such extension under development to increase insight into the learning dynamics of actor-critic learning algorithms by visualizing the optimization landscape.

Committee:

Jeff Schneider (Advisor)

Deepak Pathak

David Held

Benjamin Eysenbach