A Multisensory Approach to Present Phonemes as Language through a Wearable Haptic Device

Nathan Dunkelberger, Jennifer L. Sullivan, Joshua Bradley, Indu Manickam, Gautam Dasarathy, Richard Baraniuk, Marcia K. O'Malley

Research output: Contribution to journalArticlepeer-review

18 Scopus citations


Communication is an important part of our daily interactions; however, communication can be hindered, either through visual or auditory impairment, or because usual communication channels are overloaded. When standard communication channels are not available, our sense of touch offers an alternative sensory modality for transmitting messages. Multi-sensory haptic cues that combine multiple types of haptic sensations have shown promise for applications, such as haptic communication, that require large discrete cue sets while maintaining a small, wearable form factor. This article presents language transmission using a multi-sensory haptic device that occupies a small footprint on the upper arm. In our approach, phonemes are encoded as multisensory haptic cues consisting of vibration, radial squeeze, and lateral skin stretch components. Participants learned to identify haptically transmitted phonemes and words after training across a four day training period. A subset of our participants continued their training to extend word recognition free response. Participants were able to identify words after four days using multiple choice with an accuracy of 89% and after eight days using free response with an accuracy of 70%. These results show promise for the use of multisensory haptics for haptic communication, demonstrating high word recognition performance with a small, wearable device.

Original languageEnglish (US)
Article number9141429
Pages (from-to)188-199
Number of pages12
JournalIEEE Transactions on Haptics
Issue number1
StatePublished - Jan 1 2021


  • Wearable haptics
  • language communication
  • multi-sensory haptics
  • phoneme coding
  • tactile device

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Science Applications


Dive into the research topics of 'A Multisensory Approach to Present Phonemes as Language through a Wearable Haptic Device'. Together they form a unique fingerprint.

Cite this