Using topological data analysis in social science research: Unpacking decisions and opportunities for a new method

This research paper describes a new statistical method for engineering education, Topological Data Analysis (TDA), and considers the important decisions made during analysis and their impact on the quality of the results. We also describe why this new method may provide novel ways of understanding multidimensional data for student attitudes, beliefs, and mindsets. In this paper, we discuss the considerations that researchers must understand in conducting TDA with engineering education data. In the data analysis, a researcher must choose a filtering method, clustering method, number of filter slices (n), overlap in data, and cut height (ε) for each dimension. The importance and effect on the consistency and quality of the data differ for each decision. Some have a large impact on the results of the analysis (e.g., cut height [ε]), while others have a moderate impact on the resulting map appearance but not key structural features identified (e.g., number of filter slices [n]). We illustrate these methodological decisions as well as the results of TDA and its usefulness for engineering education using data from a project investigating first-year engineering students' underlying attitudes, beliefs, and mindsets to characterize the latent diversity of these students. A paper-and-pencil survey was administered to 3,855 students at 32 ABET accredited institutions across the U.S. in Fall 2017. After cleaning the data using attention checks within the survey, 3,711 student responses were examined for validity evidence. Exploratory factor analysis (for newly developed scales) and confirmatory factor analysis (for existing scales) were conducted. The resulting factors with strong validity evidence and high variability among engineering students were used in the TDA to map students' latent diversity. The results of this map indicate six distinct data progressions as well as a sparse group of students whose responses were not similar to the majority of the dataset. This work illustrates the opportunities for using TDA and provides a discussion of the different researcher decisions that are involved in this statistical technique.