TY - JOUR
T1 - Evaluation of auto-generated distractors in multiple choice questions from a semantic network
AU - Zhang, Lishan
AU - VanLehn, Kurt
N1 - Funding Information:
This work was supported by National Science Foundation [Grant Number DRL-0910221, IIS1123823]; Bill and Melinda Gates Foundation OPP1061281; National Natural Science Foundation of China [Grant Number 61807004].
Publisher Copyright:
© 2019 Informa UK Limited, trading as Taylor & Francis Group.
PY - 2021
Y1 - 2021
N2 - Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called “distractors”). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.
AB - Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called “distractors”). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.
KW - Question generation
KW - crowd sourcing
KW - distractor evaluation
KW - item analysis
KW - multiple choice question
KW - semantic network
UR - http://www.scopus.com/inward/record.url?scp=85066906007&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85066906007&partnerID=8YFLogxK
U2 - 10.1080/10494820.2019.1619586
DO - 10.1080/10494820.2019.1619586
M3 - Article
AN - SCOPUS:85066906007
SN - 1049-4820
VL - 29
SP - 1019
EP - 1036
JO - Interactive Learning Environments
JF - Interactive Learning Environments
IS - 6
ER -