TY - GEN
T1 - Transformer-based Automatic Mapping of Clinical Notes to Specific Clinical Concepts
AU - Ganesh, Jay
AU - Bansal, Ajay
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - A significant proportion of medical errors exist in crucial medical information, and most stem from misinterpreting non-standardized clinical notes. This research compares four transformer-based models namely: BERT (Bidirectional Encoder Representations from Transformers) Base Uncased, Emilyalsentzer Bio-ClinicalBERT, RoBERTa (Robustly Optimized BERT Pre-Training Approach), and DeBERTa (Decoding-enhanced BERT with disentangled attention) to determine which among the four is the best backbone model for mapping free text in clinical notes to specific clinical concepts. Besides, the impact of context-specific embeddings on BERT was also studied to determine the need for a clinical BERT in Clinical Skills exam scoring. This research proposes the use of DeBERTa as a backbone model in patient note scoring for the United States Medical Licensing Examination (USMLE) Clinical Skills exam after comparing it with three other transformer models. Disentangled attention and enhanced mask decoder integrated into DeBERTa were credited for its high performance. Besides, the effect of meta pseudo labeling was also investigated in this research, which in turn, further enhanced DeBERTa's performance.
AB - A significant proportion of medical errors exist in crucial medical information, and most stem from misinterpreting non-standardized clinical notes. This research compares four transformer-based models namely: BERT (Bidirectional Encoder Representations from Transformers) Base Uncased, Emilyalsentzer Bio-ClinicalBERT, RoBERTa (Robustly Optimized BERT Pre-Training Approach), and DeBERTa (Decoding-enhanced BERT with disentangled attention) to determine which among the four is the best backbone model for mapping free text in clinical notes to specific clinical concepts. Besides, the impact of context-specific embeddings on BERT was also studied to determine the need for a clinical BERT in Clinical Skills exam scoring. This research proposes the use of DeBERTa as a backbone model in patient note scoring for the United States Medical Licensing Examination (USMLE) Clinical Skills exam after comparing it with three other transformer models. Disentangled attention and enhanced mask decoder integrated into DeBERTa were credited for its high performance. Besides, the effect of meta pseudo labeling was also investigated in this research, which in turn, further enhanced DeBERTa's performance.
KW - BERT
KW - BERT Base Uncased
KW - DeBERTa
KW - RoBERTa
KW - clinical BERT
KW - meta pseudo labeling
UR - http://www.scopus.com/inward/record.url?scp=85168921695&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85168921695&partnerID=8YFLogxK
U2 - 10.1109/COMPSAC57700.2023.00080
DO - 10.1109/COMPSAC57700.2023.00080
M3 - Conference contribution
AN - SCOPUS:85168921695
T3 - Proceedings - International Computer Software and Applications Conference
SP - 558
EP - 563
BT - Proceedings - 2023 IEEE 47th Annual Computers, Software, and Applications Conference, COMPSAC 2023
A2 - Shahriar, Hossain
A2 - Teranishi, Yuuichi
A2 - Cuzzocrea, Alfredo
A2 - Sharmin, Moushumi
A2 - Towey, Dave
A2 - Majumder, AKM Jahangir Alam
A2 - Kashiwazaki, Hiroki
A2 - Yang, Ji-Jiang
A2 - Takemoto, Michiharu
A2 - Sakib, Nazmus
A2 - Banno, Ryohei
A2 - Ahamed, Sheikh Iqbal
PB - IEEE Computer Society
T2 - 47th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2023
Y2 - 26 June 2023 through 30 June 2023
ER -