Blind Source Separation using Relative Newton Method combined with Smoothing Method of Multipliers

Michael Zibulevsky (mzibee.technion.ac.il)

Abstract: We study a relative optimization framework for quasi-maximum likelihood blind source separation and relative Newton method as its particular instance. The structure of the Hessian allows its fast approximate inversion. In the second part we present Smoothing Method of Multipliers (SMOM) for minimization of sum of pairwise maxima of smooth functions, in particular sum of absolute value terms. Incorporating Lagrange multiplier into a smooth approximation of max-type function, we obtain an extended notion of non-quadratic augmented Lagrangian. Our approach does not require artificial variables, and preserves the sparse structure of Hessian. Convergence of the method is further accelerated by the Frozen Hessian strategy. We demonstrate efficiency of this approach on an example of blind separation of sparse sources. The non-linearity in this case is based on the absolute value function, which provides super-efficient source separation.

Keywords: blind source separation, maximum likelihood, Newton method, augmented Lagrangian, method of multipliers, sparse representations

Category 1: Applications -- Science and Engineering

Category 2: Nonlinear Optimization

Citation: CCIT Report #556, Department of Electrical Engineering, Technion, Haifa, September 2005

Download: [PDF]

Entry Submitted: 09/19/2005
Entry Accepted: 09/19/2005
Entry Last Modified: 09/19/2005

Modify/Update this entry