Some aspects of parallel and distributed iterative algorithms-A survey

Dimitri P. Bertsekas, John N. Tsitsiklis

Research output: Contribution to journalArticlepeer-review

115 Scopus citations


We consider iterative algorithms of the form x:=f(x), executed by a parallel or distributed computing system. We first consider synchronous executions of such iterations and study their communication requirements, as well as issues related to processor synchronization. We also discuss the parallelization of iterations of the Gauss-Seidel type. We then consider asynchronous implementations whereby each processor iterates on a different component of x, at its own pace, using the most recently received (but possibly outdated) information on the remaining components of x. While certain algorithms may fail to converge when implemented asynchronously, a large number of positive convergence results is available. We classify asynchronous algorithms into three main categories, depending on the amount of asynchronism they can tolerate, and survey the corresponding convergence results. We also discuss issues related to their termination.

Original languageEnglish (US)
Pages (from-to)3-21
Number of pages19
Issue number1
StatePublished - Jan 1991
Externally publishedYes


  • Computational methods
  • asynchronous algorithms
  • distributed algorithms
  • distributed data processing
  • iterative methods
  • parallel algorithms
  • parallel processing

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Some aspects of parallel and distributed iterative algorithms-A survey'. Together they form a unique fingerprint.

Cite this