1
$\begingroup$

Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function.

Wikipedia uses

$${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$

to denote the error.

Is the formula (1) the correct form of 0-1 loss function?

Improve this question
$\endgroup$
5
  • $\begingroup$ There is no loss function for this particular case since this algorithm iteratively updates each weight separately. The wiki says this formula is used to stop training if it is less than a certain threshold. Take a look at this and this answers for more details. $\endgroup$
    – Aray Karjauv
    Jun 26, 2021 at 10:27
  • $\begingroup$ @ArayKarjauv Thank you. What does "this particular case" refer to? Both sources claim there is a loss function. $\endgroup$
    – JJJohn
    Jun 26, 2021 at 11:07
  • $\begingroup$ This (particular) iterative learning algorithm presented in the provided articles implies a non-differentiable loss function (usually sign) that does not quantify the predictions, but only says whether they were correct or not, so you cannot optimize it using gradient descent. Therefore, it is not a loss function in the usual sense. There is a loss function if your activation is differentiable. For instance, sigmoid. Then, the loss will be binary cross-entropy. $\endgroup$
    – Aray Karjauv
    Jun 26, 2021 at 11:59
  • $\begingroup$ I've created a chat room about perceptrons. Feel free to join it: chat.stackexchange.com/rooms/126900/… $\endgroup$
    – Aray Karjauv
    Jun 26, 2021 at 20:03
  • $\begingroup$ @ArayKarjauv I don't think that a "loss function" necessarily needs to be differentiable, although this is probably just a matter of convention. Anyway, if you have an answer to this question, feel free to write it below. Comments are meant to be temporary and to provide additional info rather than full answers. $\endgroup$
    – nbro
    Jun 29, 2021 at 15:52

0

Reset to default

You must log in to answer this question.