Optimization Online


An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems

Kim-Chuan Toh (mattohkc***at***nus.edu.sg)
Sangwoon Yun (smaysw***at***nus.edu.sg)

Abstract: The affine rank minimization problem, which consists of finding a matrix of minimum rank subject to linear equality constraints, has been proposed in many areas of engineering and science. A specific rank minimization problem is the matrix completion problem, in which we wish to recover a (low-rank) data matrix from incomplete samples of its entries. A recent convex relaxation of the rank minimization problem minimizes the nuclear norm instead of the rank of the matrix. Another possible model for the rank minimization problem is the nuclear norm regularized linear least squares problem. This regularized problem is a special case of an unconstrained nonsmooth convex optimization problem, in which the objective function is the sum of a convex smooth function with Lipschitz continuous gradient and a convex function on a set of matrices. In this paper, we propose an accelerated proximal gradient algorithm, which terminates in $O(1/\sqrt{\epsilon})$ iterations with an $\epsilon$-optimal solution, to solve this unconstrained nonsmooth convex optimization problem, and in particular, the nuclear norm regularized linear least squares problem. We report numerical results for solving large-scale randomly generated matrix completion problems. The numerical results suggest that our algorithm is efficient and robust in solving large-scale random matrix completion problems. In particular, we are able to solve random matrix completion problems with matrix dimensions up to $10^5$ each in less than 10 minutes on a modest PC.

Keywords: proximal gradient, singular value decomposition, nuclear norm minimization, matrix completion

Category 1: Convex and Nonsmooth Optimization (Convex Optimization )

Category 2: Linear, Cone and Semidefinite Programming (Semi-definite Programming )

Category 3: Applications -- Science and Engineering (Data-Mining )

Citation: Preprint, Department of Mathematics, National University of Singapore, March 2009.

Download: [PDF]

Entry Submitted: 03/27/2009
Entry Accepted: 03/27/2009
Entry Last Modified: 11/17/2009

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Programming Society