On the Cluster-aware Supervised Learning (CluSL): Frameworks, Convergent Algorithms, and Applications
Abstract: This paper proposes a cluster-aware supervised learning (CluSL) framework, which integrates the clustering analysis with supervised learning (SL). The objective of CluSL is to simultaneously find the best clusters of the data points and minimize the sum of loss functions within each cluster. This framework has many potential applications in healthcare, operations management, manufacturing, and so on. Since CluSL, in general, is nonconvex, we develop a regularized alternating projection (RAP) algorithm to solve it, where at each iteration, we penalize the distance between the current clustering solution and the one from the previous iteration. By choosing a proper penalty function, we show that each iteration of the RAP algorithm can be computed efficiently. We further prove that the proposed RAP algorithm will always converge to a stationary point within a finite number of iterations. This is the first known convergence result in cluster-aware learning literature. We further extend CluSL to the high-dimensional datasets where the number of features is more than the number of data points (e.g., image data), termed F-CluSL framework. In F-CluSL, we cluster features and minimize loss function at the same time. Similarly, to solve F-CluSL, a variant of RAP algorithm (i.e., F-RAP) is developed and proven to be convergent to an $\epsilon$-stationary point. Our numerical studies demonstrate that the proposed CluSL and F-CluSL can indeed deliver more interpretable learning results and outperform the existing ones such as random forests, convolutional neural networks, and so on, both in computational time and prediction accuracy.
Keywords: Clustering, Supervised Learning, Regularization, Alternating Projection, Globally Convergent, Feature Extraction.
Category 1: Applications -- Science and Engineering (Data-Mining )
Entry Submitted: 10/29/2019
Modify/Update this entry
|Visitors||Authors||More about us||Links|
Search, Browse the Repository
Give us feedback
|Optimization Journals, Sites, Societies|