Comparison of remote and in-person digital speech-based measures of cognition

Gabriela M. Stegmann, Shira Hahn, Julie Liss, Visar Berisha, Kimberly D. Mueller

Research output: Contribution to journalArticlepeer-review

Abstract

BACKGROUND: Neuropsychological testing requires an in-person visit with a trained administrator using standard/fixed materials. Speech-based cognitive testing on mobile devices enables more frequent and timely test administration, but head-to-head comparisons of in-person and remote versions of tests are rare. We compare responses to a well-validated task conducted under supervised (in-person) and unsupervised (remote) conditions. METHOD: We used two data sets containing Cookie Theft picture descriptions (BDAE), one collected in-person, under supervision (Wisconsin Registry for Alzheimer's Prevention (WRAP); N = 912; age = 62.2 (SD = 6.7), 70% F) and one collected remotely, without supervision (participants online; N = 93, age = 36.5 (SD = 11.5), 65% F). WRAP participants were judged to be cognitively normal, and online participants self-reported as healthy. Each participant provided one picture description, yielding 93 remote and 912 in-person transcribed descriptions. Language metrics previously used in dementia research were extracted: word count (number of words spoken), MATTR (ratio of unique words to total number of words), pronoun-to-noun ratio, and semantic relevance. Comparing MATTR, pronoun-to-noun ratio, and semantic relevance values elicited in-person and remotely is important because these characteristics have been found to be impacted by declines in cognition. Differences in word counts may reflect different levels of motivation in the two settings. RESULT: Using Cohen's d effect sizes, differences in mean word count between in-person and remote participant transcripts was negligible (112 words in-person and 118 words remote; d = .10, p = .21), the difference for semantic relevance was negligible (.19 in-person and .18 remote, scale = 0 to 1; d = .14, p = .19), the difference for MATTR was small (.75 in-person and .74 remote, scale = 0 to 1; d = .37, p<.05), and the difference for pronoun-to-noun ratio was moderate (.48 in-person and .35 remote, scale = 0 to 1; d = .56, p <.05). CONCLUSION: Results show that response length and semantic relevance of responses to the Cookie Theft picture description task are comparable under supervised, in-person and unsupervised, remote conditions. Small to moderate differences in vocabulary (MATTR) and pronoun-to-noun ratio may be due to differences in age and merit further investigation.

Original languageEnglish (US)
Pages (from-to)e056438
JournalAlzheimer's and Dementia
Volume17
DOIs
StatePublished - Dec 1 2021

ASJC Scopus subject areas

  • Clinical Neurology
  • Geriatrics and Gerontology
  • Psychiatry and Mental health
  • Cellular and Molecular Neuroscience
  • Health Policy
  • Developmental Neuroscience
  • Epidemiology

Fingerprint

Dive into the research topics of 'Comparison of remote and in-person digital speech-based measures of cognition'. Together they form a unique fingerprint.

Cite this