Questions tagged [mean-squared-error]
For questions related to the mean squared error (MSE) function, which is often used to solve regression problems.
21
questions
0
votes
2answers
156 views
In variational autoencoders, why do people use MSE for the loss?
In VAEs, we try to maximize the ELBO $\mathbb(E_q log\ p(x|z) + D_{KL}(q(z|x), p(z))$), but I see that many implement the first term as MSE of the image and it's reconstruction. Is this mathematically ...
0
votes
1answer
35 views
"Porpoising" in latter stages of validation loss and MSE charts in Keras
Performing a prediction of a continuous y target using Keras, the simple structure of the code revolves around;
...
0
votes
0answers
25 views
Should an increased learning rate for an adaptive linear neuron (ADALINE) reduce the square error at every epoch?
I am completely new to neural networks and therefore, my query may have some basic conceptual problem.
I am following Fundamentals of Neural Networks by Laurene Fusett. In this book, the author ...
0
votes
0answers
49 views
Why is the error curve of a neural network trained with MSE to output $\frac{3 I_1 + 5 I_2}{2}$ given inputs $I_1$ and $I_2$ oscillating weirdly?
I just "finished" my first AI program. I programmed in Excel VBA, and I think it works well. I was checking every formula and the whole algorithm several times to make sure every formula is ...
0
votes
1answer
56 views
What error should I use for RNN?
I'm relatively new to machine learning, and I don't know what error I should use for an RNN.
I want to use a simple Elman RNN to predict the cases of Covid-19 there will be in a hospital for the next ...
2
votes
1answer
136 views
Would either $L_1$ or $L_2$ regularisation lower the MSE on the training and test data?
Consider linear regression. The mean squared error (MSE) is 120.5 for the training dataset. We've reached the minimum for the training data.
Is it possible that by applying Lasso (L1 regularization) ...
1
vote
1answer
32 views
How do I prove that the MSE is zero when all predictions are equal to the corresponding labels?
In the back-propogation algorithm, the error term is:
$$
E=\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2,
$$
where $\hat{y}_k$ is a vector of outputs from the network, $y_k$ is the vector of correct labels (...
0
votes
1answer
135 views
What is the meaning of these equations in Noise2Noise paper?
I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia.
What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to ...
-1
votes
1answer
81 views
What is the definition of a loss function in the context of neural networks?
I have read what the loss function is, but I am not sure if I have understood it. For each neuron in the output layer, the loss function is usually equal to the square of the difference value of the ...
4
votes
2answers
694 views
What is the advantage of using cross entropy loss & softmax?
I am trying to do the standard MNIST dataset image recognition test with a standard feed forward NN, but my network failed pretty badly. Now I have debugged it quite a lot and found & fixed some ...
1
vote
0answers
47 views
How can a learning rate that is too large cause the output of the network (and the error) to go to infinity?
It happened to my neural network, when I use a learning rate of <0.2 everything works fine, but when I try something above 0.4 I start getting "nan" errors because the output of my ...
2
votes
1answer
395 views
1
vote
0answers
53 views
How MSE should be appliead with multi target deep network?
I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)
Let us say I have a ...
2
votes
2answers
66 views
What does it mean if classification error is equal between two networks but the MSE is different?
I'm experimenting with training a feedforward neural network using a genetic algorithm and I've done a few tests using both the mean squared error and classification error functions as fitness ...
3
votes
0answers
247 views
How to implement Mean square error loss function in mini batch GD
I have a vectorized implementation of the neural network in c++. I successfully solve the classification problems of Fashion MNIST and CIFAR.
Now I am modifying my code to do the Linear regression. I ...
1
vote
4answers
1k views
Is it normal to have the root mean squared error greater on the test dataset than on the training dataset?
I am new to deep learning.
I am training a model and I am getting a root mean squared error (RMSE) greater on the test dataset than on the training dataset.
What could be the reason behind this? Is ...
2
votes
2answers
179 views
How to determine the target value when using ReLU as activation function?
Consider the following simple neural network with only one neuron.
The input is $x_1$ and $y_2$, where $-250 < x < 250$ and $-250 < y < 250$
The weights of the only neuron are $w_1$ and $...
3
votes
3answers
3k views
In which cases is the categorical cross-entropy better than the mean squared error?
In my code, I usually use the mean squared error (MSE), but the TensorFlow tutorials always use the categorical cross-entropy (CCE). Is the CCE loss function better than MSE? Or is it better only in ...
3
votes
2answers
444 views
How to express accuracy of a regression ANN that uses MSE loss function?
I have a regression MLP network with all input values between 0 and 1, and am using MSE for the loss function. The minimum MSE over the validation sample set comes to 0.019. So how to express the '...
7
votes
3answers
1k views
How is it possible that the MSE used to train neural networks with gradient descent has multiple local minima?
We often train neural networks by optimizing the mean squared error (MSE), which is an equation of a parabola $y=x^2$, with gradient descent.
We also say that weight adjustment in a neural network by ...
3
votes
1answer
138 views
Why is the "square error function" sometimes defined with the constant 1/2 and sometimes with the constant 1/m?
Depending on the source, I find people using different variations of the "squared error function". How come that be?
Here, it is defined as
$$
E_{\text {total }}=\sum \frac{1}{2}(\text {...
