TY - JOUR
T1 - Cohesion network analysis of CSCL participation
AU - Dascalu, Mihai
AU - McNamara, Danielle
AU - Trausan-Matu, Stefan
AU - Allen, Laura K.
N1 - Funding Information:
Author note We thank the students of University BPolitehnica^ of Bucharest who participated in our experiments and to Lucia Larise Stavarache for her support in processing the chat conversations. This research was partially supported by the FP7 208-212578 LTfLL project, by the 644187 RAGE H2020-ICT-2014 project, as well as by the NSF 1417997 and 1418378, and the Office of Naval Research (ONR N000141410343) grants to Arizona State University.
Publisher Copyright:
© 2017, Psychonomic Society, Inc.
PY - 2018/4/1
Y1 - 2018/4/1
N2 - The broad use of computer-supported collaborative-learning (CSCL) environments (e.g., instant messenger–chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54% of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.
AB - The broad use of computer-supported collaborative-learning (CSCL) environments (e.g., instant messenger–chats, forums, blogs in online communities, and massive open online courses) calls for automated tools to support tutors in the time-consuming process of analyzing collaborative conversations. In this article, the authors propose and validate the cohesion network analysis (CNA) model, housed within the ReaderBench platform. CNA, grounded in theories of cohesion, dialogism, and polyphony, is similar to social network analysis (SNA), but it also considers text content and discourse structure and, uniquely, uses automated cohesion indices to generate the underlying discourse representation. Thus, CNA enhances the power of SNA by explicitly considering semantic cohesion while modeling interactions between participants. The primary purpose of this article is to describe CNA analysis and to provide a proof of concept, by using ten chat conversations in which multiple participants debated the advantages of CSCL technologies. Each participant’s contributions were human-scored on the basis of their relevance in terms of covering the central concepts of the conversation. SNA metrics, applied to the CNA sociogram, were then used to assess the quality of each member’s degree of participation. The results revealed that the CNA indices were strongly correlated to the human evaluations of the conversations. Furthermore, a stepwise regression analysis indicated that the CNA indices collectively predicted 54% of the variance in the human ratings of participation. The results provide promising support for the use of automated computational assessments of collaborative participation and of individuals’ degrees of active involvement in CSCL environments.
KW - Cohesion network analysis, Computer-supported collaborative learning
KW - Cohesion-based discourse analysis
KW - Dialogism
KW - Participation evaluation
KW - Polyphonic model
UR - http://www.scopus.com/inward/record.url?scp=85017448746&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85017448746&partnerID=8YFLogxK
U2 - 10.3758/s13428-017-0888-4
DO - 10.3758/s13428-017-0888-4
M3 - Article
C2 - 28409485
AN - SCOPUS:85017448746
SN - 1554-351X
VL - 50
SP - 604
EP - 619
JO - Behavior Research Methods
JF - Behavior Research Methods
IS - 2
ER -