  


Analysis of LimitedMemory BFGS on a Class of Nonsmooth Convex Functions
Azam Asl(aa2821nyu.edu) Abstract: The limited memory BFGS (LBFGS) method is widely used for largescale unconstrained optimization, but its behavior on nonsmooth problems has received little attention. LBFGS can be used with or without ``scaling"; the use of scaling is normally recommended. A simple special case, when just one BFGS update is stored and used at every iteration, is sometimes also known as memoryless BFGS. We analyze memoryless BFGS with scaling, using any ArmijoWolfe line search, on the function $f(x) = ax^{(1)} + \sum_{i=2}^{n} x^{(i)}$, initiated at any point $x_0$ with $x_0^{(1)}\not = 0$. We show that if $a\ge 2\sqrt{n1}$, the absolute value of the normalized search direction generated by this method converges to a constant vector, and if, in addition, $a$ is larger than a quantity that depends on the Armijo parameter, then the iterates converge to a nonoptimal point $\bar x$ with $\bar x^{(1)}=0$, although $f$ is unbounded below. As we showed in previous work, the gradient method with any ArmijoWolfe line search also fails on the same function if $a\geq \sqrt{n1}$ and $a$ is larger than another quantity depending on the Armijo parameter, but scaled memoryless BFGS fails under a \emph{weaker} condition relating $a$ to the Armijo parameter than that implying failure of the gradient method. Furthermore, in sharp contrast to the gradient method, if a specific standard ArmijoWolfe bracketing line search is used, scaled memoryless BFGS fails when $a\ge 2 \sqrt{n1}$ \emph{regardless} of the Armijo parameter. Finally, numerical experiments indicate that similar results hold for scaled LBFGS with any fixed number of updates. Keywords: Category 1: Convex and Nonsmooth Optimization (Nonsmooth Optimization ) Citation: Download: [PDF] Entry Submitted: 09/29/2018 Modify/Update this entry  
Visitors  Authors  More about us  Links  
Subscribe, Unsubscribe Digest Archive Search, Browse the Repository

Submit Update Policies 
Coordinator's Board Classification Scheme Credits Give us feedback 
Optimization Journals, Sites, Societies  