-

 

 

 




Optimization Online





 

Stochastic subgradient method converges on tame functions

Damek Davis(dsd95***at***cornell.edu)
Dmitriy Drusvyatskiy(ddrusv***at***uw.edu)
Sham Kakade(sham***at***cs.washington.edu)
Jason D. Lee(jasonlee***at***marshall.usc.edu)

Abstract: This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity? We prove that the stochastic subgradient method, on any semialgebraic locally Lipschitz function, produces limit points that are all first-order stationary. More generally, our result applies to any function with a Whitney stratifiable graph. In particular, this work endows the stochastic subgradient method, and its proximal extension, with rigorous convergence guarantees for a wide class of problems arising in data science---including all popular deep learning architectures.

Keywords: subdifferential, stochastic subgradient method, differential inclusion, Lyapunov function, semialgebraic

Category 1: Convex and Nonsmooth Optimization (Nonsmooth Optimization )

Citation: 32 pages, 1 figure

Download: [PDF]

Entry Submitted: 05/25/2018
Entry Accepted: 05/26/2018
Entry Last Modified: 05/25/2018

Modify/Update this entry


  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository

 

Submit
Update
Policies
Coordinator's Board
Classification Scheme
Credits
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society