Optimization Online


Escaping local minima with derivative-free methods: a numerical investigation

Coralia Cartis(cartis***at***maths.ox.ac.uk)
Lindon Roberts(lindon.roberts***at***maths.ox.ac.uk)
Oliver Sheridan-Methven(oliver.sheridan-methven***at***maths.ox.ac.uk)

Abstract: We apply a state-of-the-art, local derivative-free solver, Py-BOBYQA, to global optimization problems, and propose an algorithmic improvement that is beneficial in this context. Our numerical findings are illustrated on a commonly-used test set of global optimization problems and associated noisy variants, and on hyperparameter tuning for a machine learning test set. As Py-BOBYQA is a model-based trust-region method, we compare mostly (but not exclusively) with other global optimization methods for which (global) models are important, such as Bayesian optimization and response surface methods; we also consider state-of-the-art representative deterministic and stochastic codes, such as DIRECT and CMA-ES. We find Py-BOBYQA to be competitive with global solvers that are provably designed for finding global optima, for all accuracy/budget regimes, in both smooth and noisy settings. In particular, Py-BOBYQA variants are best performing for smooth and multiplicative noise problems in high-accuracy regimes. As a by-product, some preliminary conclusions can be drawn on the relative performance of the global solvers we have tested with default settings.

Keywords: derivative-free optimization, global optimization, trust region methods.

Category 1: Nonlinear Optimization

Citation: Technical Report, Mathematical Institute, University of Oxford.

Download: [PDF]

Entry Submitted: 12/29/2018
Entry Accepted: 12/29/2018
Entry Last Modified: 12/29/2018

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society