Optimization Online


Gradient methods exploiting spectral properties

Yakui Huang (huangyakui2006***at***gmail.com)
Yu-Hong Dai (dyh***at***lsec.cc.ac.cn)
Xin-Wei Liu (mathlxw***at***hebut.edu.cn)
Hongchao Zhang (hozhang***at***math.lsu.edu )

Abstract: We propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73-88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method that takes a certain number of steps using the asymptotically optimal stepsize by Dai and Yang, and then follows by some short steps associated with this new stepsize. By employing one step retard of the asymptotic optimal stepsize, a nonmonotone variant of this method is also proposed. Under mild conditions, $R$-linear convergence of the proposed methods is established for minimizing quadratic functions. In addition, by combining gradient projection techniques and adaptive nonmonotone line search, we further extend those methods for general bound constrained optimization. Two variants of gradient projection methods combining with the Barzilai-Borwein stepsizes are also proposed. Our numerical experiments on both quadratic and bound constrained optimization indicate that the new proposed strategies and methods are very effective.

Keywords: gradient method; spectral property; Barizilai-Borwein methods; linear convergence; quadratic optimization; bound constrained optimization

Category 1: Convex and Nonsmooth Optimization (Convex Optimization )

Category 2: Nonlinear Optimization


Download: [PDF]

Entry Submitted: 11/09/2018
Entry Accepted: 11/09/2018
Entry Last Modified: 05/09/2019

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society