Plug-and-play supervisory control using muscle and brain signals for real-time gesture and error detection

Joseph DelPreto, Andres F. Salazar-Gomez, Stephanie Gil, Ramin Hasani, Frank H. Guenther, Daniela Rus

Research output: Contribution to journalArticlepeer-review

21 Scopus citations


Effective human supervision of robots can be key for ensuring correct robot operation in a variety of potentially safety-critical scenarios. This paper takes a step towards fast and reliable human intervention in supervisory control tasks by combining two streams of human biosignals: muscle and brain activity acquired via EMG and EEG, respectively. It presents continuous classification of left and right hand-gestures using muscle signals, time-locked classification of error-related potentials using brain signals (unconsciously produced when observing an error), and a framework that combines these pipelines to detect and correct robot mistakes during multiple-choice tasks. The resulting hybrid system is evaluated in a “plug-and-play” fashion with 7 untrained subjects supervising an autonomous robot performing a target selection task. Offline analysis further explores the EMG classification performance, and investigates methods to select subsets of training data that may facilitate generalizable plug-and-play classifiers.

Original languageEnglish (US)
Pages (from-to)1303-1322
Number of pages20
JournalAutonomous Robots
Issue number7
StatePublished - Sep 1 2020


  • EEG control
  • EMG control
  • Error-related potentials
  • Gesture detection
  • Human–robot interaction
  • Hybrid control
  • Plug-and-play supervisory control

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'Plug-and-play supervisory control using muscle and brain signals for real-time gesture and error detection'. Together they form a unique fingerprint.

Cite this