A neural network without a hidden layer is the same as just linear regression.
If I then use squared hinge loss and encoporate the l2 regularisation term, is it fair to then call this network the same as a linear SVM?
Going by this assumption, then if I need to implement a multiclass SVM, i can just have n output nodes (where n is the number of classes). Would this then be equivalent to having n number of SVMs, similar to a one-vs-rest method?
If I then wanted to encoporate a kernel into my SVM, could I then use an activation function or layer prior to the final output nodes (where I compute loss and add regularisation) which would then transfer this data into another feature plane the same as that of an SVM kernel?
This is my current hunch, but would like some confirmation or correction where my understanding is incorrect.