This paper considers optimization problems on the Stiefel manifold $X^TX=I_p$, where $X\in \mathbb{R}^{n \times p}$ is the variable and $I_p$ is the $p$-by-$p$ identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of $X$ and the null space of $X^T$. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai-Borwein (BB)-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn-Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs significantly better than some state-of-the-art algorithms for the nearest low-rank correlation matrix problem and is considerably competitive with the widely used SCF iteration for the Kohn-Sham total energy minimization.
Citation
@TECHREPORT{StiManOpt-Jiang-Dai2012, author = {Jiang, Bo and Dai, Yu-Hong}, title = {A Framework of Constraint Preserving Update Schemes for Optimization on Stiefel Manifold}, institution = {Academy of Mathematics and Systems Sciences, Chinese Academy of Sciences}, year = {2012} }
Article
View A Framework of Constraint Preserving Update Schemes for Optimization on Stiefel Manifold