Some papers on RL environment diffuculty and human perceived difficulty
I was looking for some papers on how to quantify the difficulty in RL environments as well as if there is work on how humans perceive difficulty compared to RL agents.
Here is my short, curated list on this (I did not read those in detail yet since I have other stuff to attend to, but I will do in the future):
- Hardness in Markov Decision Processes: Theory and Practice
- Surveys hardness notions for tabular RL
- Colosseum Benchmark
- Bad-Policy Density: A Measure of Reinforcement Learning Hardness
- measures fraction of policies wholse value is below a shreshold
- environment is harder when "good" policies are rarer in policy space
- On the Limits of Tabular Hardness Metrics for Deep RL: A Study with the Pharos Benchmark
- tabular hardness does not transfer cleanly to deep RL from pixels.
For Atari environments reseach has some implicit notions of difficulty but no works that treat the difficulty of a game as the primary object of stud. Also to keep in mind:
Atari-5: Distilling the Arcade Learning Environment down to Five Games
Transferable Curricula through Difficulty Conditioned Generators
- Parameterized RL environments explicitly modeling both environment difficulty and student ability (matching the difficulty of an environment to the current ability of the student)
- Parameterized Environment Response Model (PERM)
On the Perception of Difficulty: Differences between Humans and AI
- reviews how human and AI diffuculty are usually estimates separately
- we still lack a clean way to compare them consistently on the same tasks
Human Learning about AI Performance
- people tend to project human task difficulty onto AI difficulty
Machine versus Human Attention in Deep Reinforcement Learning Tasks
- compares agent saliency maps with human attention
- based on the Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
- asks how similar their visual representations are
Get in Touch
If you have any feedback you want to share with me feel free to reach out at mail@sebastianwette.de. I would be more than happy to chat about it.