Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function.

Wikipedia uses

$${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$

to denote the error.

Is the formula (1) the correct form of 0-1 loss function?

  • $\begingroup$ There is no loss function for this particular case since this algorithm iteratively updates each weight separately. The wiki says this formula is used to stop training if it is less than a certain threshold. Take a look at this and this answers for more details. $\endgroup$ Jun 26 at 10:27
  • $\begingroup$ @ArayKarjauv Thank you. What does "this particular case" refer to? Both sources claim there is a loss function. $\endgroup$
    – JJJohn
    Jun 26 at 11:07
  • $\begingroup$ This (particular) iterative learning algorithm presented in the provided articles implies a non-differentiable loss function (usually sign) that does not quantify the predictions, but only says whether they were correct or not, so you cannot optimize it using gradient descent. Therefore, it is not a loss function in the usual sense. There is a loss function if your activation is differentiable. For instance, sigmoid. Then, the loss will be binary cross-entropy. $\endgroup$ Jun 26 at 11:59
  • $\begingroup$ I've created a chat room about perceptrons. Feel free to join it: chat.stackexchange.com/rooms/126900/… $\endgroup$ Jun 26 at 20:03
  • $\begingroup$ @ArayKarjauv I don't think that a "loss function" necessarily needs to be differentiable, although this is probably just a matter of convention. Anyway, if you have an answer to this question, feel free to write it below. Comments are meant to be temporary and to provide additional info rather than full answers. $\endgroup$
    – nbro
    Jun 29 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.