Optimization Online


Hedge algorithm and Dual Averaging schemes

Michel Baes(michel.baes***at***ifor.math.ethz.ch)
Michael Bürgisser(michael.buergisser***at***ifor.math.ethz.ch)

Abstract: We show that the Hedge algorithm, a method that is widely used in Machine Learning, can be interpreted as a particular instance of Dual Averaging schemes, which have recently been introduced by Nesterov for regret minimization. Based on this interpretation, we establish three alternative methods of the Hedge algorithm: one in the form of the original method, but with optimal parameters, one that requires less a priori information, and one that is better adapted to the context of the Hedge algorithm. All our modified methods have convergence results that are better or at least as good as the performance guarantees of the vanilla method. In numerical experiments, our methods significantly outperform the original scheme.

Keywords: Hedge algorithm, Dual Averaging schemes, Black-Box optimization, portfolio optimization

Category 1: Convex and Nonsmooth Optimization (Convex Optimization )

Citation: Technical Report, ETH Zurich, December 2011

Download: [PDF]

Entry Submitted: 12/06/2011
Entry Accepted: 12/06/2011
Entry Last Modified: 12/06/2011

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society