Emerging non-volatile memory (eNVM) based synaptic devices are attractive for the replacement of SRAM in the hardware implementation of artificial neural networks (ANNs). However, one of the critical challenges for eNVM is the reliability concerns due to data retention and write endurance failures. This paper investigates the impact of these two failures in the multilayer perceptron (MLP) using our developed NeuroSim+ simulator. For the retention failure in offline classification, we consider various possible conductance drift scenarios and the reported physical model based on conductance variation. The results confirm that faster degradation on the classification accuracy is highly correlated with larger deviation in the weighted sum. For the endurance failure in online learning, the strength of conductance tuning is assumed to become weaker over write pulse cycles. The analysis suggests that the learning accuracy is less impacted because the network is able to adapt itself and activate more synapses to participate in the weight update when the tuning capability of synapses are degraded.