Topological Knowledge Distillation for Wearable Sensor Data

Eun Som Jeon, Hongjun Choi, Ankita Shukla, Yuan Wang, Matthew P. Buman, Pavan Turaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations


Converting wearable sensor data to actionable health insights has witnessed large interest in recent years. Deep learning methods have been utilized in and have achieved a lot of successes in various applications involving wearables fields. However, wearable sensor data has unique issues related to sensitivity and variability between subjects, and dependency on sampling-rate for analysis. To mitigate these issues, a different type of analysis using topological data analysis has shown promise as well. Topological data analysis (TDA) captures robust features, such as persistence images (PI), in complex data through the persistent homology algorithm, which holds the promise of boosting machine learning performance. However, because of the computational load required by TDA methods for large-scale data, integration and implementation has lagged behind. Further, many applications involving wearables require models to be compact enough to allow deployment on edge-devices. In this context, knowledge distillation (KD) has been widely applied to generate a small model (student model), using a pre-trained high-capacity network (teacher model). In this paper, we propose a new KD strategy using two teacher models - one that uses the raw time-series and another that uses persistence images from the time-series. These two teachers then train a student using KD. In essence, the student learns from heterogeneous teachers providing different knowledge. To consider different properties in features from teachers, we apply an annealing strategy and adaptive temperature in KD. Finally, a robust student model is distilled, which utilizes the time series data only. We find that incorporation of persistence features via second teacher leads to significantly improved performance. This approach provides a unique way of fusing deep-learning with topological features to develop effective models.

Original languageEnglish (US)
Title of host publication56th Asilomar Conference on Signals, Systems and Computers, ACSSC 2022
EditorsMichael B. Matthews
PublisherIEEE Computer Society
Number of pages6
ISBN (Electronic)9781665459068
StatePublished - 2022
Event56th Asilomar Conference on Signals, Systems and Computers, ACSSC 2022 - Virtual, Online, United States
Duration: Oct 31 2022Nov 2 2022

Publication series

NameConference Record - Asilomar Conference on Signals, Systems and Computers
ISSN (Print)1058-6393


Conference56th Asilomar Conference on Signals, Systems and Computers, ACSSC 2022
Country/TerritoryUnited States
CityVirtual, Online


  • knowledge distillation
  • time series data analysis
  • topological data analysis
  • wearable sensor data

ASJC Scopus subject areas

  • Signal Processing
  • Computer Networks and Communications


Dive into the research topics of 'Topological Knowledge Distillation for Wearable Sensor Data'. Together they form a unique fingerprint.

Cite this