It is known that the conjugate-gradient algorithm is at least as good as the steepest-descent algorithm for minimizing quadratic functions. It is shown here that the conjugate-gradient algorithm is actually superior to the steepest-descent algorithm in that, in the generic case, at each iteration it yields a lower cost than does the steepest-descent algorithm, when both start at the same point.
Key Words Convergence of optimization algorithms - steepest-descent methods - conjugate-gradient methods
Communicated by D. Q. Mayne
Thanks are due to Professor R. W. Sargent, Imperial College, London, England, for suggestions concerning presentation.