When determining the decision boundary to separate datapoints that belong to two classes, multiple decision boundaries can be found that correctly classify the samples. A decision boundary that falls equally between the separation in the data has the least risk of causing an error. Support Vector Machines determine the margin between the decision boundary and the nearest datapoint of both classes and then optimizes the boundary to be equidistant away from sample of both classes.

Image by notebook author.

Although SVM by default uses a linear model and therefore linearly classifies data, the kernel trick can be used for non-linear classification. By using the Radial Basis Function (RBF), polynomial function, or sigmoid function, the data are mapped to a higher dimensional space which allows their non-linear separation.

Image From: Murphy, K. P. (2021). Figure 14.4. In Machine Learning: A Probabilistic Perspective. textbook, MIT Press.