Optimization Online


On spectral properties of steepest descent methods

Roberta De Asmundis (roberta.deasmundis***at***uniroma1.it)
Daniela di Serafino (daniela.diserafino***at***unina2.it)
Filippo Riccio (filippo.riccio***at***unina2.it)
Gerardo Toraldo (toraldo***at***unina.it)

Abstract: In recent years it has been made more and more clear that the critical issue in gradient methods is the choice of the step length, whereas using the gradient as search direction may lead to very effective algorithms, whose surprising behaviour has been only partially explained, mostly in terms of the spectrum of the Hessian matrix. On the other hand, the convergence of the classical Cauchy steepest descent (CSD) method has been extensively analysed and related to the spectral properties of the Hessian matrix, but the connection with the spectrum of the Hessian has been little exploited to modify the method in order to improve its behaviour. In this work we show how, for convex quadratic problems, moving from some theoretical properties of the CSD method, second-order information provided by the step length can be exploited to dramatically improve the usually poor practical behaviour of this method. This allows to achieve computational results comparable with those of the Barzilai and Borwein algorithm, with the further advantage of a monotonic behaviour.

Keywords: steepest descent methods, quadratic optimization, Hessian spectral properties

Category 1: Nonlinear Optimization

Category 2: Nonlinear Optimization (Quadratic Programming )


Download: [PDF]

Entry Submitted: 02/10/2012
Entry Accepted: 02/10/2012
Entry Last Modified: 05/29/2014

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society