/Community Regularization of Visually Grounded Dialog

Community Regularization of Visually Grounded Dialog

Akshat Agarwal, Swaminathan Gurumurthy, Vasu Sharma, Mike Lewis and Katia Sycara
Conference Paper, Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), May, 2019

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


The task of conducting visually grounded dialog involves learning goal-oriented cooperative dialog between autonomous agents who exchange information about a scene through several rounds of questions and answers in natural language. We posit that requiring artificial agents to adhere to the rules of human language, while also requiring them to maximize information exchange through dialog is an ill-posed problem. We observe that humans do not stray from a common language because they are social creatures who live in communities, and have to communicate with many people everyday, so it is far easier to stick to a common language even at the cost of some efficiency loss. Using this as inspiration, we propose and evaluate a multi-agent community-based dialog framework where each agent interacts with, and learns from, multiple agents, and show that this community-enforced regularization results in more relevant and coherent dialog (as judged by human evaluators) without sacrificing task performance (as judged by quantitative metrics).

BibTeX Reference
author = {Akshat Agarwal and Swaminathan Gurumurthy and Vasu Sharma and Mike Lewis and Katia Sycara},
title = {Community Regularization of Visually Grounded Dialog},
booktitle = {Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019)},
year = {2019},
month = {May},
publisher = {IFAAMAS},
keywords = {Visual Dialog; Multi Agent Reinforcement Learning; Curriculum Learning; Emergent Communication},