Optimization Online


Separable Approximations and Decomposition Methods for the Augmented Lagrangian

Rachael Tappenden( R.Tappenden***at***ed.ac.uk )
Peter Richtarik(peter.richtarik***at***ed.ac.uk)
Burak Buke(b.buke***at***ed.ac.uk)

Abstract: In this paper we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian. In particular, we study and compare the Diagonal Quadratic Approximation Method (DQAM) of Mulvey and Ruszczy\'{n}ski and the Parallel Coordinate Descent Method (PCDM) of Richt\'{a}rik and Tak\'{a}\v{c}. We show that the two methods are equivalent for feasibility problems up to the selection of a single step-size parameter. Furthermore, we prove an improved complexity bound for PCDM under strong convexity, and show that this bound is at least $8(L'/\bar{L})(\omega-1)^2$ times better than the best known bound for DQAM, where $\omega$ is the degree of partial separability and $L'$ and $\bar{L}$ are the maximum and average of the block Lipschitz constants of the gradient of the quadratic penalty appearing in the augmented Lagrangian.

Keywords: augmented Lagrangian, decomposition, diagonal quadratic approximation, DQA, expected separable overapproximation, ESO, parallel coordinate descent method, PCDM, convex optimization, parallel computing

Category 1: Convex and Nonsmooth Optimization (Convex Optimization )

Category 2: Convex and Nonsmooth Optimization (Nonsmooth Optimization )

Category 3: Stochastic Programming


Download: [PDF]

Entry Submitted: 08/30/2013
Entry Accepted: 08/30/2013
Entry Last Modified: 08/30/2013

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society