Is $(y_i - \hat y_i)x_i$, part of the formula for updating weights for perceptron, the gradient of some kind of loss function?

Question

A post gives a formula for perceptron to update weights

I understand almost all the parts of it, except for the part $(y_i - \hat y_i)x_i$ where does it come from? Is it the gradient of some kind of loss function? If yes, what is the definition of the loss function?

The OP seems doesn't give the hypothesis, so that $\hat y_i = h(x_i)$

However, this hypothesis seems prevalent

\begin{align} \hat{y} &= sign(\mathbf{w} \cdot \mathbf{x} + b) \tag{1}\\ &= sign({w}_{1}{x}_{1}+{w}_{2}{x}_{2} + ... + w_nx_n + b) \\ \end{align}

where

$$ sign(z) = \begin{cases} 1, & z \ge 0 \\ -1, & z < 0 \end{cases} $$

How do I get $(y_i - \hat y_i)x_i$ from function (1)

Answer 1

It depends on your hypothesis $h$. The author of the original article compares the dot product with a threshold:

So for a binary classification problem $h$ can be defined as follows:

$$ h = \begin{cases} 1 & \text{if $f>z$}\\ 0 & \text{otherwise} \end{cases} $$

That is, $\hat y_i$ is your prediction and $\hat y_i = h(x_i)$, $y_i$ is a real label and $x_i$ is a sample.

Finally, you can update your weghts $w_n = w_n + \eta(y_i - \hat y_i)x_i$, where $n$ is a number of the weight and $i$ denotes a number of the label/sample pair.

Depending on how you define your hypothesis you will have a different optimization algorithm. Take a look at this answer for more details.