Odo: Design of multimodal chatbot for an experiential media system

Ravi Bhushan; Karthik Kulkarni; Vishal Kumar Pandey; Connor Rawls; Brandon Mechtley; Suren Jayasuriya; Christian Ziegler

doi:10.3390/mti4040068

Odo: Design of multimodal chatbot for an experiential media system

Ravi Bhushan, Karthik Kulkarni, Vishal Kumar Pandey, Connor Rawls, Brandon Mechtley, Suren Jayasuriya, Christian Ziegler

Arts, Media and Engineering, School of (AME)

Research output: Contribution to journal › Article › peer-review

8 Scopus citations

Abstract

This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.

Original language	English (US)
Article number	68
Pages (from-to)	1-16
Number of pages	16
Journal	Multimodal Technologies and Interaction
Volume	4
Issue number	4
DOIs	https://doi.org/10.3390/mti4040068
State	Published - Dec 2020

Keywords

Chatbots
Digital theater
Media arts
Performing arts

ASJC Scopus subject areas

Neuroscience (miscellaneous)
Human-Computer Interaction
Computer Science Applications
Computer Networks and Communications

Access to Document

10.3390/mti4040068

Cite this

@article{b72bf4670078422d8a00df9229d2ed4f,

title = "Odo: Design of multimodal chatbot for an experiential media system",

abstract = "This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.",

keywords = "Chatbots, Digital theater, Media arts, Performing arts",

author = "Ravi Bhushan and Karthik Kulkarni and Pandey, {Vishal Kumar} and Connor Rawls and Brandon Mechtley and Suren Jayasuriya and Christian Ziegler",

note = "Publisher Copyright: {\textcopyright} 2020 by the authors. Licensee MDPI, Basel, Switzerland.",

year = "2020",

month = dec,

doi = "10.3390/mti4040068",

language = "English (US)",

volume = "4",

pages = "1--16",

journal = "Multimodal Technologies and Interaction",

issn = "2414-4088",

publisher = "MDPI AG",

number = "4",

}

TY - JOUR

T1 - Odo

T2 - Design of multimodal chatbot for an experiential media system

AU - Bhushan, Ravi

AU - Kulkarni, Karthik

AU - Pandey, Vishal Kumar

AU - Rawls, Connor

AU - Mechtley, Brandon

AU - Jayasuriya, Suren

AU - Ziegler, Christian

PY - 2020/12

Y1 - 2020/12

N2 - This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.

AB - This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.

KW - Chatbots

KW - Digital theater

KW - Media arts

KW - Performing arts

UR - http://www.scopus.com/inward/record.url?scp=85092211485&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85092211485&partnerID=8YFLogxK

U2 - 10.3390/mti4040068

DO - 10.3390/mti4040068

M3 - Article

AN - SCOPUS:85092211485

SN - 2414-4088

VL - 4

SP - 1

EP - 16

JO - Multimodal Technologies and Interaction

JF - Multimodal Technologies and Interaction

IS - 4

M1 - 68

ER -

Odo: Design of multimodal chatbot for an experiential media system

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this