Open loop stability criterion for layered and fully-connected neural networks

Stability is considered for a broad range of neural network taxonomies via the open loop interconnection matrix, A. We discuss the morphism between stochastic or deterministic, analog or digital, fully-connected Hopfield or the sparse arrays of cellular automata using a model based on control theory. We first review the connection between the state-transition matrix, the interconnection matrix and the nonlinearities. A linear transformation of the model yields an equivalent representation where the nonlinear processing element, or activation function, is transferred from after the summing operation to before the summing operation. Sufficient contraints on the weight matrix and the activation function are specified to ensure stability for both analog and digital networks.