Questions tagged [validation]

For questions related to the validation of a machine learning model, which is different from testing a model, which is done after training. Validation is usually performed for early stopping (i.e. assess when the model is over-fitting) or hyper-parameter optimization during training. However, often people use (either correctly/intensionally or not) the terms "validation" and "testing" interchangeably, so context needs to be taken into account.

Filter by
Sorted by
Tagged with
-1 votes
1 answer

how to decide the optimum model?

I have split the database available into 70% training, 15% validation, and 15% test, using holdout validation. I have trained the model and got the following results: training accuracy 100%, ...
user avatar
0 votes
0 answers

What are the standard ways to measure the quality of a set of numerical predictions that include uncertainties?

I have a radial basis function that supplies uncertainties (standard deviations) with its predictions, which are numerical values. This function is computed for a particular point by computing its ...
user avatar
  • 101
0 votes
0 answers

Why would my neural network have either an accuracy of 90% or 10% on the validation data, given a random initialization?

I'm making a custom neural network framework (in C++, if that is of any help). When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or ...
user avatar
10 votes
3 answers

Should I choose a model with the smallest loss or highest accuracy?

I have two Machine Learning models (I use LSTM) that have a different result on the validation set (~100 samples data): Model A: Accuracy: ~91%, Loss: ~0.01 Model B: Accuracy: ~83%, Loss: ~0.003 The ...
user avatar
  • 2,551
2 votes
0 answers

Which evaluation metrics should be used in training, validation and testing of a model?

Which specific performance evaluation metrics are used in training, validation, and testing, and why? I am thinking error metrics (RMSE, MAE, MSE) are used in validation, and testing should use a wide ...
user avatar