Support Vector Machine (SVM)

Support Vector Machines find the decision boundary that maximizes the distance (margin) to the closest training examples from each class. This maximum-margin principle provides strong theoretical guarantees about generalization and produces classifiers that are robust to overfitting, especially in high-dimensional spaces.

The kernel trick extends SVMs to nonlinear problems by implicitly mapping data into higher-dimensional spaces where linear separation is possible. The RBF (Radial Basis Function) kernel is most popular, effectively measuring similarity between points and enabling complex, curved decision boundaries without explicitly computing the high-dimensional mapping.

SVMs were dominant in machine learning before deep learning and remain relevant for specific use cases: small to medium datasets where they often outperform neural networks, high-dimensional data like text classification and genomics where their performance scales well, and situations requiring strong theoretical guarantees. For most production growth applications today, gradient boosting has largely replaced SVMs for tabular data, and transformers have replaced them for text, but SVMs remain a useful tool in the ML practitioner's toolkit for specific scenarios.