Optimization Online


Accelerated Stochastic Peaceman-Rachford Method for Empirical Risk Minimization

Jianchao Bai(jianchaobai***at***nwpu.edu.cn)
Fengmiao Bian(bianfm17***at***sjtu.edu.cn)
Xiaokai Chang(xkchang***at***lut.cn)
Lin Du(lindu***at***nwpu.edu.cn)

Abstract: This work is devoted to studying an Accelerated Stochastic Peaceman-Rachford Splitting Method (AS-PRSM) for solving a family of structural empirical risk minimization problems. The objective function of this problem is the sum of a possibly nonsmooth convex function and a finite-sum of smooth convex component functions. The smooth subproblem in AS-PRSM is solved by an stochastic gradient methods using variance reduction techniques and accelerated techniques, while the possibly nonsmooth subproblem is solved by introducing an indefinite proximal term to transform it into a proximal mapping. By a proper choice for the algorithmic parameters, we show that AS-PRSM converges in expectation with a sublinear convergence rate in terms of the objective gap and the constraint violation. Preliminary experiments on testing the popular graph-guided fused lasso problem in machine learning and the 3D CT reconstruction problem in medical image processing show that the proposed AS-PRSM is very efficient and promising.

Keywords: empirical risk minimization, convex optimization, stochastic Peaceman-Rachford method, indefinite proximal term, complexity

Category 1: Stochastic Programming

Category 2: Convex and Nonsmooth Optimization (Convex Optimization )

Citation: 17 pages

Download: [PDF]

Entry Submitted: 08/17/2021
Entry Accepted: 08/17/2021
Entry Last Modified: 08/17/2021

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society