TY - JOUR
T1 - Wav2ddk
T2 - Analytical and clinical validation of an automated diadochokinetic rate estimation algorithm on remotely collected speech
AU - Kadambi, Prad
AU - Stegmann, Gabriela M.
AU - Liss, Julie
AU - Berisha, Visar
AU - Hahn, Shira
N1 - Publisher Copyright:
© 2023 American Speech-Language-Hearing Association.
PY - 2023/8
Y1 - 2023/8
N2 - Purpose: Oral diadochokinesisis a useful task in assessment ofspeech motor func-tion in the context of neurological disease. Remote collection of speech tasks pro-vides a convenient alternative to in-clinic visits, but scoring these assessments can be a laborious process for clinicians. This work describes Wav2DDK, an automated algorithm for estimating the diadochokinetic (DDK) rate on remotely collected audio from healthy participants and participants with amyotrophic lateral sclerosis(ALS). Method: Wav2DDK was developed using a corpus of 970 DDK assessments from healthy and ALS speakers where ground truth DDK rates were provided manually by trained annotators. The clinical utility of the algorithm was demonstrated on a corpus of 7,919 assessments collected longitudinally from 26 healthy controls and 82 ALS speakers. Corpora were collected via the participants’ own mobile device, and instructions for speech elicitationwereprovidedvia amobile app.DDK rate was estimated by parsing the character transcript from a deep neural network transformer acoustic model trained on healthy and ALS speech. Results: Algorithm estimated DDK rates are highly accurate, achieving.98 cor-relation with manual annotation, and an average error of only 0.071 syllables per second. The rate exactly matched ground truth for 83% of files and was within 0.5 syllables per second for 95% of files. Estimated rates achieve a high test-retest reliability (r =.95) and show good correlation with the revised ALS functional rating scale speech subscore (r =.67). Conclusion: We demonstrate a system for automated DDK estimation that increases efficiency of calculation beyond manual annotation. Thorough analyti-cal and clinical validation demonstrates that the algorithm is not only highly accurate, but also provides a convenient, clinically relevant metric for tracking longitudinal decline in ALS, serving to promote participation and diversity of participants in clinical research.
AB - Purpose: Oral diadochokinesisis a useful task in assessment ofspeech motor func-tion in the context of neurological disease. Remote collection of speech tasks pro-vides a convenient alternative to in-clinic visits, but scoring these assessments can be a laborious process for clinicians. This work describes Wav2DDK, an automated algorithm for estimating the diadochokinetic (DDK) rate on remotely collected audio from healthy participants and participants with amyotrophic lateral sclerosis(ALS). Method: Wav2DDK was developed using a corpus of 970 DDK assessments from healthy and ALS speakers where ground truth DDK rates were provided manually by trained annotators. The clinical utility of the algorithm was demonstrated on a corpus of 7,919 assessments collected longitudinally from 26 healthy controls and 82 ALS speakers. Corpora were collected via the participants’ own mobile device, and instructions for speech elicitationwereprovidedvia amobile app.DDK rate was estimated by parsing the character transcript from a deep neural network transformer acoustic model trained on healthy and ALS speech. Results: Algorithm estimated DDK rates are highly accurate, achieving.98 cor-relation with manual annotation, and an average error of only 0.071 syllables per second. The rate exactly matched ground truth for 83% of files and was within 0.5 syllables per second for 95% of files. Estimated rates achieve a high test-retest reliability (r =.95) and show good correlation with the revised ALS functional rating scale speech subscore (r =.67). Conclusion: We demonstrate a system for automated DDK estimation that increases efficiency of calculation beyond manual annotation. Thorough analyti-cal and clinical validation demonstrates that the algorithm is not only highly accurate, but also provides a convenient, clinically relevant metric for tracking longitudinal decline in ALS, serving to promote participation and diversity of participants in clinical research.
UR - http://www.scopus.com/inward/record.url?scp=85168243835&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85168243835&partnerID=8YFLogxK
U2 - 10.1044/2023_JSLHR-22-00282
DO - 10.1044/2023_JSLHR-22-00282
M3 - Article
C2 - 37556308
AN - SCOPUS:85168243835
SN - 1092-4388
VL - 66
SP - 3166
EP - 3181
JO - Journal of Speech, Language, and Hearing Research
JF - Journal of Speech, Language, and Hearing Research
IS - 8S
ER -