The idea with imitation learning is to train agents by imitating human behavior. If the agent imitates the human sufficiently well, its behavior would be aligned. See ยง7.1 of this paper for more information.
With imitation learning, I think the goal is to produce aligned behavior that is as capable as a human (but not more capable). I think this corresponds to the first step of iterated distillation and amplification, namely distilling the human behavior.
| Title | Publication date | Author | Publisher | Affected organizations | Affected people | Affected agendas | Notes |
|---|---|---|---|---|---|---|---|
| Scalable agent alignment via reward modeling: a research direction | 2018-11-19 | Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, Shane Legg | arXiv | Google DeepMind | Recursive reward modeling, Imitation learning, inverse reinforcement learning, Cooperative inverse reinforcement learning, myopic reinforcement learning, iterated amplification, debate | This paper introduces the (recursive) reward modeling agenda, discussing its basic outline, challenges, and ways to overcome those challenges. The paper also discusses alternative agendas and their relation to reward modeling. |