-

 

 

 




Optimization Online





 

Fast convergence of inertial dynamics and algorithms with asymptotic vanishing damping

H Attouch(hedy.attouch***at***univ-montp2.fr)
Z Chbani(chbaniz***at***uca.ma)
J Peypouquet(juan.peypouquet***at***usm.cl)
P Redont(patrick.redont***at***univ-montp2.fr)

Abstract: In a Hilbert space setting $\mathcal H$, we study the fast convergence properties as $t \to + \infty$ of the trajectories of the second-order differential equation \begin{equation*} \ddot{x}(t) + \frac{\alpha}{t} \dot{x}(t) + \nabla \Phi (x(t)) = g(t), \end{equation*} where $\nabla\Phi$ is the gradient of a convex continuously differentiable function $\Phi: \mathcal H \to \mathbb R$, $\alpha$ is a positive parameter, and $g: [t_0, + \infty[ \rightarrow \mathcal H$ is a {\em small} perturbation term. In this inertial system, the viscous damping coefficient $\frac{\alpha}{t}$ vanishes asymptotically, but not too rapidly. For $\alpha \geq 3$, and $\int_{t_0}^{+\infty} t \|g(t)\| dt < + \infty$, just assuming that $\argmin \Phi \neq \emptyset$, we show that any trajectory of the above system satisfies the fast convergence property \begin{align*} \Phi(x(t))- \min_{\mathcal H}\Phi \leq \frac{C}{t^2}. \end{align*} Moreover, for $\alpha > 3$, we show that any trajectory converges weakly to a minimizer of $\Phi$. The strong convergence is established in various practical situations. These results complement the $\mathcal O(t^{-2})$ rate of convergence for the values obtained by Su, Boyd and Cand\`es in the unperturbed case $g=0$. Time discretization of this system, and some of its variants, provides new fast converging algorithms, expanding the field of rapid methods for structured convex minimization introduced by Nesterov, and further developed by Beck and Teboulle with FISTA. This study also complements recent advances due to Chambolle and Dossal.

Keywords: Convex optimization, fast convergent methods, dynamical systems, gradient flows, inertial dynamics, vanishing viscosity, Nesterov method

Category 1: Convex and Nonsmooth Optimization (Convex Optimization )

Citation:

Download: [PDF]

Entry Submitted: 10/28/2015
Entry Accepted: 10/28/2015
Entry Last Modified: 10/28/2015

Modify/Update this entry


  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository

 

Submit
Update
Policies
Coordinator's Board
Classification Scheme
Credits
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society