New answers tagged

1

Mean Square Error (MSE) is a quadratic function and the further you go away from your optimum the bigger (quadratic) the MSE gets. Take $o_{expected}=20$ and $o_{net}=40$ as example. Your MSE is then 400, because $MSE = (o_{expected}-o_{net})^2$. Just imagine $y = x^2$ with x being the output of your network. If you want to shift the parabola with optimum at ...


0

Chiming in because I had the same question and stumbled across your post. It seems like the general version of your question still has not been answered. In general, a well-formed gradient update rule is all you need to be able to train the network. We are thinking of converting to a "loss function" because that is the typical flow in the structure ...


0

I think I understand the process now. Let $x$ has the dimension $(n, p)$ and W has the dimension $(p, q)$. In neural network, $n$ denotes the samples in the batch and $p, q$ denotes input dimension and output dimension of each layer, respectively. We don't use bias $b$ here just for simplicity. I had trouble in understanding the process of differentiation ...


Top 50 recent answers are included