Odo: Design of multimodal chatbot for an experiential media system

Ravi Bhushan, Karthik Kulkarni, Vishal Kumar Pandey, Connor Rawls, Brandon Mechtley, Suren Jayasuriya, Christian Ziegler

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.

Original languageEnglish (US)
Article number68
Pages (from-to)1-16
Number of pages16
JournalMultimodal Technologies and Interaction
Issue number4
StatePublished - Dec 2020


  • Chatbots
  • Digital theater
  • Media arts
  • Performing arts

ASJC Scopus subject areas

  • Neuroscience (miscellaneous)
  • Human-Computer Interaction
  • Computer Science Applications
  • Computer Networks and Communications


Dive into the research topics of 'Odo: Design of multimodal chatbot for an experiential media system'. Together they form a unique fingerprint.

Cite this