The Robotics Institute

Carnegie Mellon Robotics Institute

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

Performance of a cognitive personal assistant, RADAR, consisting of multiple machine learning components, natural language processing, and optimization was examined with a test explicitly developed to measure the impact of integrated machine learning when used by a human user in a real world setting. Three conditions (conventional tools, Radar without learning, and Radar with learning) were evaluated in a large-scale, between-subjects study. The study revealed that integrated machine learning does produce a positive impact on overall performance. This paper also discusses how specific machine learning components contributed to human-system performance.

Notes

Sponsor: Defense Advanced Research Projects Agency (DARPA)

Grant ID: Contract No. NBCHD030010

Associated Center(s) / Consortia: Vision and Autonomous Systems Center, Quality of Life Technology Center, and Center for Integrated Manfacturing Decision Systems

Associated Lab(s) / Group(s): Intelligent Coordination and Logistics Laboratory and Human-Robot Interaction Group

Associated Project(s): Reflective Agents with Distributed Adaptive Reasoning

Number of pages: 7

Text Reference

Aaron Steinfeld, S. Rachael Bennett, Kyle Cunningham, Matt Lahut, Pablo-Alejandro Quinones, Django Wexler, Daniel Siewiorek, Jordan Hayes, Paul Cohen, Julie Fitzgerald, Othar Hansson, Mike Pool, and Mark Drummond, "Evaluation of an Integrated Multi-Task Machine Learning System with Humans in the Loop," NIST Performance Metrics for Intelligent Systems Workshop (PerMIS), 2007.

BibTeX Reference

@inproceedings{Steinfeld_2007_5838,
   author = "Aaron Steinfeld and S. Rachael Bennett and Kyle Cunningham and Matt Lahut and Pablo-Alejandro Quinones and Django Wexler and Daniel Siewiorek and Jordan Hayes and Paul Cohen and Julie Fitzgerald and Othar Hansson and Mike Pool and Mark Drummond",
   title = "Evaluation of an Integrated Multi-Task Machine Learning System with Humans in the Loop",
   booktitle = "NIST Performance Metrics for Intelligent Systems Workshop (PerMIS)",
   year = "2007",
}

The Robotics Institute

Menu