Optimal Use of Verbal Instructions for Multi-robot Human Navigation Guidance

Harel Yedidsion, Jacqueline Deans, Connor Sheehan, Mahathi Chillara, Justin Hart, Peter Stone, Raymond J. Mooney

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations


Efficiently guiding humans in indoor environments is a challenging open problem. Due to recent advances in mobile robotics and natural language processing, it has recently become possible to consider doing so with the help of mobile, verbally communicating robots. In the past, stationary verbal robots have been used for this purpose at Microsoft Research, and mobile non-verbal robots have been used at UT Austin in their multi-robot human guidance system. This paper extends that mobile multi-robot human guidance research by adding the element of natural language instructions, which are dynamically generated based on the robots’ path planner, and by implementing and testing the system on real robots. Generating natural language instructions from the robots’ plan opens up a variety of optimization opportunities such as deciding where to place the robots, where to lead humans, and where to verbally instruct them. We present experimental results of the full multi-robot human guidance system and show that it is more effective than two baseline systems: one which only provides humans with verbal instructions, and another which only uses a single robot to lead users to their destinations.

Original languageEnglish (US)
Title of host publicationSocial Robotics - 11th International Conference, ICSR 2019, Proceedings
EditorsMiguel A. Salichs, Shuzhi Sam Ge, Emilia Ivanova Barakova, John-John Cabibihan, Alan R. Wagner, Álvaro Castro-González, Hongsheng He
Number of pages11
ISBN (Print)9783030358877
StatePublished - 2019
Externally publishedYes
Event11th International Conference on Social Robotics, ICSR 2019 - Madrid, Spain
Duration: Nov 26 2019Nov 29 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11876 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference11th International Conference on Social Robotics, ICSR 2019


  • Human robot interaction
  • Indoor navigation
  • Multi robot coordination
  • Natural language

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Optimal Use of Verbal Instructions for Multi-robot Human Navigation Guidance'. Together they form a unique fingerprint.

Cite this