Optimization Online


On the Cluster-aware Supervised Learning (CluSL): Frameworks, Convergent Algorithms, and Applications

Shutong Chen (cst950928***at***hotmail.com)
Weijun Xie (wxie***at***vt.edu)

Abstract: This paper proposes a cluster-aware supervised learning (CluSL) framework, which integrates the clustering analysis with supervised learning (SL). The objective of CluSL is to simultaneously find the best clusters of the data points and minimize the sum of loss functions within each cluster. This framework has many potential applications in healthcare, operations management, manufacturing, and so on. Since CluSL, in general, is nonconvex, we develop a regularized alternating minimization (RAM) algorithm to solve it, where at each iteration, we penalize the distance between the current clustering solution and the one from the previous iteration. By choosing a proper penalty function, we show that each iteration of the RAM algorithm can be computed efficiently. We further prove that the proposed RAM algorithm will always converge to a stationary point within a finite number of iterations. This is the first known convergence result in cluster-aware learning literature. Furthermore, we extend CluSL to the high-dimensional datasets, termed F-CluSL framework. In F-CluSL, we cluster features and minimize loss function at the same time. Similarly, to solve F-CluSL, a variant of RAM algorithm (i.e., F-RAM) is developed and proven to be convergent to an $\epsilon$-stationary point. Our numerical studies demonstrate that the proposed CluSL and F-CluSL can outperform the existing ones such as random forests and support vector classification, both in the interpretability of learning results and prediction accuracy.

Keywords: Clustering, Supervised Learning, Regularization, Alternating Projection, Globally Convergent, Feature Extraction.

Category 1: Applications -- Science and Engineering (Data-Mining )


Download: [PDF]

Entry Submitted: 10/29/2019
Entry Accepted: 10/29/2019
Entry Last Modified: 10/20/2020

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society