Transfer Learning based Deep Encoder Decoder Network for Amyloid PET Harmonization with Small Datasets

Background: The existence of multiple amyloid tracers with varied characteristics poses a significant challenge to standardized interpretation and quantification of amyloid PET data. We previously demonstrated that a deep learning model using a residual inception encoder-decoder network (RIED-Net) architecture improved harmonization of florbetapir (FBP) and PiB PET. However, scarcity of head-to-head comparison datasets of other amyloid PET tracer may limit the ability to generalize this approach to the other amyloid PET tracers. In this research, we investigate the performance of RIED-Net model trained on smaller datasets and explore transfer learning (TL) approaches which can leverage multiple different datasets and address the data scarcity issue. Method: We previously trained our RIED-Net model using PiB-FBP data from Open Access Series of Imaging Studies (OASIS) (N=92) and tested the performance in the independent Global Alzheimer’s Association Interactive Network (GAAIN) (N=46) dataset. We now use the same procedure to train the RIED-Net model using the smaller GAAIN dataset and evaluate the model performance. We then examine the feasibility of a TL approach by comparing the model performance between RIED-Net model directly trained on the PiB-florbetaben (FBB) from GAAIN (N=35) and the TL approach in which we started with the optimal RIED-Net model for PiB-FBP data and fine-tuned the network parameters using the GAAIN PiB-FBB data. A cross-validation (CV) approach was used to evaluate model performance based on difference before and after harmonization in terms of correlation between mean cortical SUVR (mcSUVR) measurements from different tracers. Result: RIED-Net model trained using the smaller GAAIN PiB-FBP data improved the agreement of mcSUVR measurements as measured by Pearson correlation from r=0.93 to r=0.96 (p<0.01) on test-set. For PiB-FBB harmonization, the TL approach provided numerically better agreement in mcSUVR from r=0.96 (direct train) to r=0.97 (with TL), demonstrating the potential benefits of applying TL based technique to address the issue of smaller sample size. Conclusion: We demonstrate our RIED-Net model can achieve improved harmonization even when trained on a relatively small training set. We also demonstrate the potential of transferring knowledge learned from one tracer pairs to improve the harmonization model of other related tracer pairs.