Questions tagged [validation]

For questions related to the validation of a machine learning model, which is different from testing a model, which is done after training. Validation is usually performed for early stopping (i.e. assess when the model is over-fitting) or hyper-parameter optimization during training. However, often people use (either correctly/intensionally or not) the terms "validation" and "testing" interchangeably, so context needs to be taken into account.

Filter by
Sorted by
Tagged with
0
votes
0answers
52 views

What are the standard ways to measure the quality of a set of numerical predictions that include uncertainties?

I have a radial basis function that supplies uncertainties (standard deviations) with its predictions, which are numerical values. This function is computed for a particular point by computing its ...
0
votes
0answers
49 views

Why would my neural network have either an accuracy of 90% or 10% on the validation data, given a random initialization?

I'm making a custom neural network framework (in C++, if that is of any help). When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or ...
9
votes
3answers
2k views

Should I choose a model with the smallest loss or highest accuracy?

I have two Machine Learning models (I use LSTM) that have a different result on the validation set (~100 samples data): Model A: Accuracy: ~91%, Loss: ~0.01 Model B: Accuracy: ~83%, Loss: ~0.003 The ...