This article presents a novel formulation designed to improve the robustness of Support Vector Machines (SVMs) when dealing with class imbalance and noisy data. Unlike the conventional Soft Margin SVM, which penalizes the magnitude of constraint violations, the proposed model quantifies the number of violations and aims to minimize their frequency. To achieve this, a binary variable is incorporated into the objective function of the primal SVM formulation, replacing the traditional slack variable. Furthermore, each misclassified instance is assigned a priority and an associated constraint. The resulting formulation is a mixed-integer programming model, efficiently solved using a heuristic algorithm based on Benders decomposition. Experiments were conducted on twenty public datasets, encompassing both binary and multiclass classification tasks, using four kernel types: linear, polynomial, sigmoid, and radial basis function. The performance of the proposed model was benchmarked against Soft Margin SVM, Weighted SVM, and NuSVC. The proposed model exhibited several interesting properties, including improved tolerance to label and feature noise in the training and testing phases, a decision boundary shift favoring the minority class, a reduced number of support vectors, and faster inference speed. The open-source Python implementation of the proposed SVM model is available.

Keywords