Optimization Online


Hybrid MPI/OpenMP parallel support vector machine training

Kristian Woodsend(k.j.woodsend***at***sms.ed.ac.uk)
Jacek Gondzio(J.Gondzio***at***ed.ac.uk)

Abstract: Support Vector Machines are a powerful machine learning technology, but the training process involves a dense quadratic optimization problem and is computationally challenging. A parallel implementation of Support Vector Machine training has been developed, using a combination of MPI and OpenMP. Using an interior point method for the optimization and a reformulation that avoids the dense Hessian matrix, the structure of the augmented system matrix is exploited to partition data and computations amongst parallel processors efficiently. The new implementation has been applied to solve problems from the PASCAL Challenge on Large Scale Learning. We show that our approach is competitive, and is able to solve problems in the Challenge many times faster than other parallel approaches. We also demonstrate that the hybrid version performs more efficiently than the version using pure MPI.

Keywords: SVM training, hybrid parallelism, largescale, interior point method

Category 1: Applications -- Science and Engineering (Data-Mining )

Category 2: Nonlinear Optimization (Quadratic Programming )

Category 3: Optimization Software and Modeling Systems (Parallel Algorithms )

Citation: Technical Report ERGO 09-001, January 6, 2009, School of Mathematics and Maxwell Institute for Mathematical Sciences, University of Edinburgh, James Clerk Maxwell Building, The King's Buildings, Edinburgh, EH9 3JZ, UK

Download: [PDF]

Entry Submitted: 01/12/2009
Entry Accepted: 01/12/2009
Entry Last Modified: 01/12/2009

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Programming Society