  


Augmented L1 and NuclearNorm Models with a Globally Linearly Convergent Algorithm
MingJun Lai (mjlaimath.uga.edu) Abstract: This paper studies the longexisting idea of adding a nice smooth function to "smooth" a nondifferentiable objective function in the context of sparse optimization, in particular, the minimization of $x_1+1/(2\alpha)x_2^2$, where $x$ is a vector, as well as those of the minimization of $X_*+1/(2\alpha)X_F^2$, where $X$ is a matrix and $X_*$ and $X_F$ are the nuclear and Frobenius norms of $X$, respectively. We show that they can efficiently recover sparse vectors and lowrank matrices. In particular, they enjoy exact and stable recovery guarantees similar to those known for minimizing $x_1$ and $X_*$ under the conditions on the sensing operator such as its nullspace property, restricted isometry property, spherical section property, or RIPless property. To recover a (nearly) sparse vector $x^0$, minimizing $x_1+1/(2\alpha)x_2^2$ returns (nearly) the same solution as minimizing $x_1$ almost whenever $\alpha\ge 10x^0_\infty$. The same relation also holds between minimizing $X_*+1/(2\alpha)X_F^2$ and minimizing $X_*$ for recovering a (nearly) lowrank matrix $X^0$, if $\alpha\ge 10X^0_2$. Furthermore, we show that the linearized Bregman algorithm for minimizing $x_1+1/(2\alpha)x_2^2$ subject to $Ax=b$ enjoys global linear convergence as long as a nonzero solution exists, and we give an explicit rate of convergence. The convergence property does not require a sparse solution or any properties on $A$. To our knowledge, this is the best known global convergence result for firstorder sparse optimization algorithms. The Matlab codes and demos of this approach, including the original, line search, and Nesterov acceleration versions, can be found from the second authorâ€™s homepage. Keywords: sparse optimization, compressed sensing, low rank matrix, matrix recovery, global linear convergence Category 1: Convex and Nonsmooth Optimization (Nonsmooth Optimization ) Citation: Rice CAAM Report TR201202 Download: [PDF] Entry Submitted: 01/22/2012 Modify/Update this entry  
Visitors  Authors  More about us  Links  
Subscribe, Unsubscribe Digest Archive Search, Browse the Repository

Submit Update Policies 
Coordinator's Board Classification Scheme Credits Give us feedback 
Optimization Journals, Sites, Societies  