  


New Computational Guarantees for Solving Convex Optimization Problems with First Order Methods, via a Function Growth Condition Measure
Robert M. Freund (rfreundmit.edu) Abstract: Motivated by recent work of Renegar, we present new computational methods and associated computational guarantees for solving convex optimization problems using firstorder methods. Our problem of interest is the general convex optimization problem f^* = \min_{x \in Q} f(x), where we presume knowledge of a strict lower bound f_slb < f^*. [Indeed, f_slb is naturally known when optimizing many loss functions in statistics and machine learning (leastsquares, logistic loss, exponential loss, total variation loss, etc.) as well as in Renegar's transformed version of the standard conic optimization problem; in all these cases one has f_slb = 0 < f^*.] We introduce a new functional measure called the growth constant G for f(.), that measures how quickly the level sets of f(.) grow relative to the function value, and that plays a fundamental role in the complexity analysis. When f(.) is nonsmooth, we present new computational guarantees for the Subgradient Descent Method and for smoothing methods, that can improve existing computational guarantees in several ways, most notably when the initial iterate x^0 is far from the optimal solution set. When f(.) is smooth, we present a scheme for periodically restarting the Accelerated Gradient Method that can also improve existing computational guarantees when x^0 is far from the optimal solution set, and in the presence of added structure we present a scheme using parametrically increased smoothing that further improves the associated computational guarantees. Keywords: convex optimization, firstorder methods, nonsmooth optimization, computational complexity Category 1: Convex and Nonsmooth Optimization (Convex Optimization ) Category 2: Convex and Nonsmooth Optimization (Nonsmooth Optimization ) Citation: MIT Operations Research Center Working Paper, MIT, September 2015. Download: [PDF] Entry Submitted: 09/16/2015 Modify/Update this entry  
Visitors  Authors  More about us  Links  
Subscribe, Unsubscribe Digest Archive Search, Browse the Repository

Submit Update Policies 
Coordinator's Board Classification Scheme Credits Give us feedback 
Optimization Journals, Sites, Societies  