Optimization Online


A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization

Andre Milzarek(andremilzarek***at***bicmr.pku.edu.cn)
Xiantao Xiao(xtxiao***at***dlut.edu.cn)
Shicong Cen(tsen9731***at***pku.edu.cn)
Zaiwen Wen(wenzw***at***pku.edu.cn)
Michael Ulbrich(mulbrich***at***ma.tum.de)

Abstract: In this work, we present a globalized stochastic semismooth Newton method for solving stochastic optimization problems involving smooth nonconvex and nonsmooth convex terms in the objective function. We assume that only noisy gradient and Hessian information of the smooth part of the objective function is available via calling stochastic first and second order oracles. The proposed method can be seen as a hybrid approach combining stochastic semismooth Newton steps and stochastic proximal gradient steps. Two inexact growth conditions are incorporated to monitor the convergence and the acceptance of the semismooth Newton steps and it is shown that the algorithm converges globally to stationary points in expectation. Moreover, under standard assumptions and utilizing random matrix concentration inequalities, we prove that the proposed approach locally turns into a pure stochastic semismooth Newton method and converges r-superlinearly with high probability. We present numerical results and comparisons on $\ell_1$-regularized logistic regression and nonconvex binary classification that demonstrate the efficiency of our algorithm.

Keywords: nonsmooth stochastic optimization, stochastic approximation, semismooth Newton method, stochastic second order information, global and local convergence

Category 1: Convex and Nonsmooth Optimization (Nonsmooth Optimization )


Download: [PDF]

Entry Submitted: 03/09/2018
Entry Accepted: 03/09/2018
Entry Last Modified: 03/09/2018

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society