Optimization Online


Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold

Shixiang Chen (sxchen***at***se.cuhk.edu.hk)
Shiqian Ma (sqma***at***math.ucdavis.edu)
Anthony Man-Cho So (manchoso***at***se.cuhk.edu.hk)
Tong Zhang (tongzhang***at***tongzhang-ml.org)

Abstract: We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objective function and thus tend to converge slowly in practice. Algorithms in the second class are proximal point algorithms, which involve subproblems that can be as difficult as the original problem. Algorithms in the third class are based on operator-splitting techniques, but they usually lack rigorous convergence guarantees. In this paper, we propose a retraction-based proximal gradient method for solving this class of problems. We prove that the proposed method globally converges to a stationary point. Iteration complexity for obtaining an $\epsilon$-stationary solution is also analyzed. Numerical results on solving sparse PCA and compressed modes problems are reported to demonstrate the advantages of the proposed method.

Keywords: Manifold Optimization; Nonsmooth; Proximal Gradient Method; Iteration Complexity; Semi-smooth Newton Method; Stiefel Manifold; Sparse PCA; Compressed Modes

Category 1: Nonlinear Optimization


Download: [PDF]

Entry Submitted: 11/02/2018
Entry Accepted: 11/02/2018
Entry Last Modified: 05/10/2019

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society