Emergent Communication and Decision-Making in Multi-Agent Teams

Abstract

Explicit communication among humans is key to coordinating and learning. In multi-agent reinforcement learning for partially-observable environments, agents may convey information to others via learned communication, allowing the team to complete its task. However, agents need to be able to communicate more than simply referential messages about their observations. Agents must use communication to coordinate their actions to effectively accomplish their goals. This thesis argues that sparse emergent communication in multi-agent teams is essential for agents to encompass general decision-making prowess and fully reach potential in decentralized and social settings. First, I show the previous issues with emergent communication through the lens of interfacing between humans and groups of agents. Through human experiments, I find that humans learn to work best with agent partners which use discrete communication tokens with continuous encodings and a sparse message rate. An interpretability analysis shows that the tokens that work best with humans have the best representation capacity. Then, I investigate the usage of autoencoders to increase the representational capacity of observations. These results further confirm that sparser communication can be enabled without any loss of performance strictly based on intrinsic messaging objectives through mutual information and the information bottleneck. Lastly, I explore the development of language and communication through a social learning lens. In order to understand the minimal amount of communication, one needs to understand how communication may arise, especially in decentralized systems and teams where new agents are added without prior experience. Together, these techniques allow for sparse, intelligent communication between agents and groups of agents with a human partner with strong representational properties that allow for low empirical sample complexity and the potential to learn in social scenarios.