2

This is very easy to prove. Let's first prove that, if $\hat{y}_k = y_k$, then the $E = 0$. I will leave all steps, so that it's super clear. \begin{align} E &=\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2 \\ &=\frac{1}{2}\sum_k(y_k - y_k)^2\\ &=\frac{1}{2}\sum_k(0)^2\\ &=\frac{1}{2}\sum_k 0\\ &=\frac{1}{2} 0\\ &=0\\ \end{align} To prove the ...


2

We sometimes see that binary cross-entropy (BCE) loss is used for regression problems. This post is my opinion on using BCE for regression problems. The figure below is the plots of BCE, $-t*\log(x) - (1-t)*\log(1-x)$, for several target values $t = 0.0, 0.1, ..., 0.5$. (The plots for $t>0.5$ are mirror images of those for $t<0.5$, so I omitted them.) ...


2

On page 5 of the VAE paper, it's clearly stated We let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary data) whose distribution parameters are computed from $\mathbf{z}$ with a MLP (a fully-connected neural network with a single hidden layer, see appendix $\mathrm{...


1

To provide a good answer would fill several pages. To keep it very simple try many different loss functions on your model. Your goal is to have the highest performance based on some desired prediction metric (e.g., RMSE, MAE, MAPE, etc.). You almost always have plenty of time to try many loss functions so you don't need to have a full understanding, and ...


1

The answer is largely the same whether we consider $\ell_1$ or $\ell_2$ regularisation, so I will just speak generally about regularisation. Mean square error for training data Given some training data $\{(x_i, y_i)\}_{i = 1}^n$, a linear regression line $Y = aX + b$ fit using the least squares method looks for coefficients that minimise the sum of squares, ...


Only top voted, non community-wiki answers of a minimum length are eligible