Part 1

SVM begins with a geometric choice: which separator is safest?

Before optimization, SVM is a simple visual idea: separate the classes while staying as far as possible from the closest points.

Many separating lines

If all these lines classify the training data correctly, which one feels most reliable?

SVM prefers the boundary that has the largest safety gap from nearby points.

How do we write a separating line in a way that also works for many dimensions?

In two dimensions, the boundary can be written as:

\[ w_1x_1+w_2x_2+b=0 \]

In many dimensions:

\[ w^Tx+b=0 \]

The vector \(w\) controls orientation. The number \(b\) shifts the boundary.

Prediction comes from which side of the hyperplane the point lies on.

If \(w^Tx+b\) is positive for a point, which class should it belong to?

\[ f(x)=w^Tx+b \qquad \hat{y}=\operatorname{sign}(f(x)) \]

Should a correctly classified point very close to the boundary be treated as fully safe?

SVM wants a confident separation. The closest points determine the safety gap. These closest points are support vectors.

\[ \text{support vectors} = \text{training points closest to the boundary} \]

Far-away points usually do not decide the SVM boundary. The closest points do.

How is SVM different from logistic regression if both can learn a linear boundary?

Model	Main focus	Output style
Logistic Regression	learn probabilities using log loss	probability-like output
SVM	maximize margin using hinge loss	decision score; probabilities need extra calibration