The Majorization Approach to SVM: The SVMMaj Package in R

Support Vector Machines (SVMs) have gained considerable popularity over the last two decades for binary classification. This paper concentrates on a recent optimization approach to SVMs, the SVM majorization approach, or SVM-Maj for short. This method is aimed at small and medium sized Support Vector Machine (SVM) problems, in which SVM-Maj performs well relative to other solvers. To obtain an SVM solution, most other solvers need to solve the dual problem. In contrast, SVM-Maj solves the primal SVM optimization iteratively thereby converging to the SVM solution. Furthermore, the simplicity of SVM-Maj makes it intuitively more accessible to the researcher than the state-of-art decomposition methods. Moreover, SVM-Maj can easily handle any well-behaved error function, while the traditional SVM solvers focus particularly on the absolute-hinge error. In this paper, the SVM-Maj approach is enhanced to include the use of different kernels, the standard way in the SVM literature for handling nonlinearities in the predictor space. In addition, we introduce the R package SVMMaj that implements this methodology. Amongst its features are the weighting of the error for individual objects in the training dataset, handling nonlinear prediction through monotone spline transformations and through kernels, and functions to do cross validation.