-

 

 

 




Optimization Online





 

CONJUGATE GRADIENT WITH SUBSPACE OPTIMIZATION

Sahar Karimi(s2karimi***at***uwaterloo.ca)
Stephen Vavasis(vavasis***at***uwaterloo.ca)

Abstract: In this paper we present a variant of the conjugate gradient (CG) algorithm in which we invoke a subspace minimization subproblem on each iteration. We call this algorithm CGSO for "conjugate gradient with subspace optimization". It is related to earlier work by Nemirovsky and Yudin. We apply the algorithm to solve unconstrained strictly convex problems. As with other CG algorithms, the update step on each iteration is a linear combination of the last gradient and last update. Unlike some other conjugate gradient methods, our algorithm attains a theoretical complexity bound of $O(\sqrt{\frac{L}{l}}\log(\frac{1}{\epsilon})$, where the ratio $\frac{L}{l}$ characterizes the strong convexity of the objective function. In practice, CGSO competes with other CG-type algorithms by incorporating some second order information in each iteration.

Keywords: Conjugate Gradient, First Order Algorithms, Convex OPtimization

Category 1: Nonlinear Optimization (Unconstrained Optimization )

Category 2: Convex and Nonsmooth Optimization (Convex Optimization )

Citation:

Download: [PDF]

Entry Submitted: 02/06/2012
Entry Accepted: 02/06/2012
Entry Last Modified: 02/06/2012

Modify/Update this entry


  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository

 

Submit
Update
Policies
Coordinator's Board
Classification Scheme
Credits
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society