TY - JOUR
T1 - Impact of Non-Ideal Characteristics of Resistive Synaptic Devices on Implementing Convolutional Neural Networks
AU - Sun, Xiaoyu
AU - Yu, Shimeng
N1 - Funding Information:
Manuscript received April 26, 2019; revised June 26, 2019; accepted July 29, 2019. Date of publication August 5, 2019; date of current version September 17, 2019. This work was supported in part by the NSF under Grant CCF-1903951 and in part by the ASCENT, one of the SRC/DARPA JUMP Centers. This article was recommended by Guest Editor R. Joshi. (Corresponding author: Xiaoyu Sun.) The authors are with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA (e-mail: xsun377@gatech.edu; shimeng.yu@ece.gatech.edu).
Publisher Copyright:
© 2011 IEEE.
PY - 2019/9
Y1 - 2019/9
N2 - Emerging non-volatile memory (eNVM) based resistive synaptic devices have shown great potential for implementing deep neural networks (DNNs). However, the eNVM devices typically suffer from various non-ideal effects which may degrade the performance of the system. Based on a representative convolutional neural network (CNN) model for CIFAR-10 dataset, this paper comprehensively investigates the impact of those non-ideal characteristics, such as nonlinearity and asymmetry of conductance tuning, variations, endurance and retention, on the training/inference accuracy. The compact models of the device non-ideal effects are incorporated into the TensorFlow framework. Our simulation results suggest that 1) the training accuracy is more sensitive to the asymmetry of conductance tuning than the nonlinearity; 2) the conductance range variation does not degrade the training accuracy, instead, a small variation can even reduce the accuracy loss introduced by asymmetry; 3) device-to-device variation can also remedy the accuracy loss due to asymmetry while cycle-to-cycle variation leads to significant accuracy degradation; 4) the accuracy degradation will not be noticeable if the endurance cycles are more than 7,000 cycles; and 5) different drifting modes affect the inference accuracy differently, and the best case is where the conductance is drifting up/down randomly.
AB - Emerging non-volatile memory (eNVM) based resistive synaptic devices have shown great potential for implementing deep neural networks (DNNs). However, the eNVM devices typically suffer from various non-ideal effects which may degrade the performance of the system. Based on a representative convolutional neural network (CNN) model for CIFAR-10 dataset, this paper comprehensively investigates the impact of those non-ideal characteristics, such as nonlinearity and asymmetry of conductance tuning, variations, endurance and retention, on the training/inference accuracy. The compact models of the device non-ideal effects are incorporated into the TensorFlow framework. Our simulation results suggest that 1) the training accuracy is more sensitive to the asymmetry of conductance tuning than the nonlinearity; 2) the conductance range variation does not degrade the training accuracy, instead, a small variation can even reduce the accuracy loss introduced by asymmetry; 3) device-to-device variation can also remedy the accuracy loss due to asymmetry while cycle-to-cycle variation leads to significant accuracy degradation; 4) the accuracy degradation will not be noticeable if the endurance cycles are more than 7,000 cycles; and 5) different drifting modes affect the inference accuracy differently, and the best case is where the conductance is drifting up/down randomly.
KW - Emerging non-volatile memory
KW - deep neural networks
KW - in-situ training
KW - inference
KW - reliability
KW - synaptic devices
KW - variation
UR - http://www.scopus.com/inward/record.url?scp=85070706192&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85070706192&partnerID=8YFLogxK
U2 - 10.1109/JETCAS.2019.2933148
DO - 10.1109/JETCAS.2019.2933148
M3 - Article
AN - SCOPUS:85070706192
SN - 2156-3357
VL - 9
SP - 570
EP - 579
JO - IEEE Journal on Emerging and Selected Topics in Circuits and Systems
JF - IEEE Journal on Emerging and Selected Topics in Circuits and Systems
IS - 3
M1 - 8787884
ER -