Emergent Dominance Hierarchies in Reinforcement Learning Agents

Modern Reinforcement Learning (RL) algorithms are able to outperform humans in a wide variety of tasks. Multi-agent reinforcement learning (MARL) settings present additional challenges around cooperation in mixed-motive groups. Social conventions and norms, often inspired by human institutions, are used as tools for striking the balance between individual and group objectives. We examine a fundamental social convention that underlies cooperation in animal and human societies: dominance hierarchies. We adapt the ethological theory of dominance hierarchies to artificial agents, borrowing established terminology and definitions. We provide an environment we call Chicken Coop, and we demonstrate that populations of RL agents in that environment can invent, learn, enforce, and transmit a dominance hierarchy to new populations. The dominance hierarchies that emerge in it have a similar structure to those studied in chickens, mice, fish, and other species.