Overview
A common method to understand policies of RL agents that operate on a visual space is to just watch it act. Working with RL policies based on non-visual sensor data poses a unique problem though, because humans are unable to evaluate it. This motivates us to create the novel OccupancyViz library. It allows human-interpretable real time visualization of the polar occupancy grid portions of states that the RL agent takes as input, and their corresponding actions. Our library also includes an interactive visual tool for manipulating polar occupancy grids to let the user easily experiment with how different states change the behavior of the agent.
The potential benefits of this visualization are that looking at such a visualization while seeing the agent move in the world could make debugging many issues dramatically easier. It also give users an easier lens to interpret the policy of an agent, by easily seeing how it responds to many environments that they fully understand, and seeing how it responds in terms of what the robot sees, as opposed to just watching it in the world. In contexts where the cost of failure is very high, this can give users a great deal more confidence in the polices of the robot.
Software
(Coming Soon)