Optimization Online


Object-Parallel Infrastructure for Implementing First-Order Methods, with an Example Application to LASSO

Jonathan Eckstein (jeckstei***at***rci.rutgers.edu)
Gyorgy Matyasfalvi (matyasfalvi***at***gmail.com)

Abstract: We describe the design of a C++ vector-manipulation substrate that allows first-order optimization algorithms to be expressed in a concise and readable manner, yet still achieve high performance in parallel computing environments. We use standard object-oriented techniques of encapsulation and operator overloading, combined with a novel “symbolic temporaries” delayed-evaluation system that greatly reduces the overhead induced by compiler temporaries and economizes on memory references. We also provide infrastructure to support line-search methods by caching function values and gradients at previously-visited points in a transparent manner that does not “clutter” the principal implementation. We demonstrate the usefulness of our vector-substrate tools by employing them to efficiently solve large-scale LASSO problems using hundreds of processor cores. We reformulate the LASSO problem as a bound-constrained quadratic optimization, and then solve it using the Spectral Projected Gradient (SPG) method implemented through our vector-manipulation substrate.

Keywords: Parallel computing, first-order methods, LASSO

Category 1: Optimization Software and Modeling Systems (Parallel Algorithms )

Category 2: Nonlinear Optimization (Bound-constrained Optimization )

Category 3: Nonlinear Optimization (Unconstrained Optimization )

Citation: RUTCOR, Rutgers University

Download: [PDF]

Entry Submitted: 01/22/2015
Entry Accepted: 01/22/2015
Entry Last Modified: 04/03/2015

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society