Support Vector Machine Convert optimisation problem from argmax to argmin

Question

I'm new to the AI Stackexchange and wasn't certain if this should go here or to Maths instead but thought the context with ML may be useful to understand my problem. I hope posting this question here could help another student learning about Support Vector Machines some day.

I'm currently learning about Support Vector Machines at university and came across a weird step I could not understand. We were talking about basic SVMs and formulated the optimisation problem $\max_{w,b} \{ \frac{1}{||w||} \min_n(y^{(n)}f(x^{(n)}))\}$ which we then simplified down to $\max_{w,b} \{ \frac{1}{||w||}\}$ by introducing $\kappa$ as a scaling factor for $w$ and $b$ according to the margin of the SVM. Now our lecturer converted it without explanation into a quadratic optimisation problem as $\min_{w,b}\{\frac{1}{2} ||w||^2\}$ which I could not explain myself. I hope someone with context can help me how this is possible and what math or trick is behind this approach?

---

Notation information:

• $w$ - weight matrix
• $b$ - bias (sometimes denoted $w_0$ I believe?)
• $x^{(n)}$ - Independent variable (vector)
• $y^{(n)}$ - Dependent variable (scalar classifying the input in a binary classifcation as $y=1$ or $y=-1$)

Thank you very much!

Answer 1

So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion.

Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{||w||}$ decreases, the overall value increases, i.e. we maximise it. The reason for choosing $\frac{1}{2}||w||^2$ turns out to be a less mathematic and more practical one as our optimisation algorithms used perform better on quadratic functions and the $\frac{1}{2}$ seems to be a rather arbitrary scaling choice.

If anyone has something to add, I'd love to hear any details though!