SVM chooses a boundary that is not just correct, but safely separated.
Support Vector Machine is a margin-based classifier. It asks: among all possible boundaries, which one leaves the widest safety gap from the closest training points?
The core question
Opening question
If three different lines classify all training points correctly, which line should we trust on future data?
SVM prefers the line with the largest margin. The margin is the safety gap between the decision boundary and the closest examples from each class.
The closest points are called support vectors because they decide where the boundary sits.
SVM is geometric: boundary, margin, and support vectors.
Session story
The pages follow the same flow as the notebook: visual intuition first, then formulas, then code-ready practical choices.
Roadmap
Geometry
Separating lines, hyperplanes, margins, and support vectors.
Scratch SVM
Hinge loss, gradients, updates, and a small from-scratch classifier.
Soft margin
Slack variables, C, violations, and class weights.
Kernels
Nonlinear data, feature maps, polynomial kernel, RBF kernel, and gamma.
Practice
Scaling, pipelines, tuning, probability estimates, multiclass, and checklist.
SVR
Use SVM ideas for regression with an epsilon-insensitive tube.
Where SVM is useful
| Use case | Why SVM can work well |
|---|---|
| Text classification | High-dimensional sparse features often work well with linear SVM. |
| Medical classification | Strong margin-based boundaries can work well on small to medium numeric datasets. |
| Bioinformatics | Often many features and fewer samples, where margin-based models can be strong. |
| Engineered image features | SVM can classify embeddings or handcrafted visual features. |
| Nonlinear toy/medium data | RBF and polynomial kernels can create curved boundaries. |
| Regression with robust tolerance | SVR fits a function while ignoring small errors inside an epsilon tube. |