MetaGenRL: Enhancing Generalization in Meta RL 🌟

Meta RL is good at adaptation to very similar environments. But can we meta-learn general RL algorithms to replace human engineered ones?
Our new approach MetaGenRL is able to.

Presented at ICLR 2020, spotlight talk.
Paper: https://arxiv.org/abs/1910.04098
Blog post: http://louiskirsch.com/metagenrl

This research was conducted at the Swiss AI Lab IDSIA under the supervision of Jürgen Schmidhuber.

My website: http://louiskirsch.com/
Jürgen's website: http://people.idsia.ch/~juergen/

Full abstract:

Biological evolution has distilled the experiences of many learners into the general learning algorithms of humans. Our novel meta reinforcement learning algorithm MetaGenRL is inspired by this process. MetaGenRL distills the experiences of many complex agents to meta-learn a low-complexity neural objective function that decides how future individuals will learn. Unlike recent meta-RL algorithms, MetaGenRL can generalize to new environments that are entirely different from those used for meta-training. In some cases, it even outperforms human-engineered RL algorithms. MetaGenRL uses off-policy second-order gradients during meta-training that greatly increase its sample efficiency.