Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [validation]

For questions related to the validation of a machine learning model, which is different from testing a model, which is done after training. Validation is usually performed for early stopping (i.e. assess when the model is over-fitting) or hyper-parameter optimization during training. However, often people use (either correctly/intensionally or not) the terms "validation" and "testing" interchangeably, so context needs to be taken into account.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Optimal number of epochs for Transformer network on time series data?

I have a transformer network that is trained on time series data. The task is to predict if a variable will increase a certain percentage in the next dt days. The input is data from the 90 previous ...
QCQCQC's user avatar
  • 101
-1 votes
1 answer
44 views

Is model order of a model class (for example, polynomial regression class) a hyperparameter or a tuning parameter?

We know that in ML we have tuning parameters and hyperparameters. Is model order of a model class (for example, polynomial regression class) a hyperparameter or a tuning parameter?
DSPinfinity's user avatar
0 votes
0 answers
7 views

If not chosen all the data in the train partition, is it still k-fold cross validation?

I have a dataset of 900 images, distributed across 6 classes, with 150 images per class. To develop a classifier and assess its performance, I will utilize k-fold cross-validation. In this case, I ...
noone's user avatar
  • 123
1 vote
0 answers
45 views

Why is my validation accuracy fluctuating between two inverse values?

I am currently going through the FastAI course and to practise, I wanted to code a neural network that classifies the FashionMNIST dataset from scratch. Lately, I've been running into an issue where I ...
DerOeko's user avatar
  • 13
0 votes
0 answers
19 views

Very high ACC (ca. 95%) with 1DConvNet for Time Series

Does this sound legit, for people working with CNN and Time Series? I have a Framework that applies Dynamic Tim Warping (DTW) on time series, using the DTW distance matrix, I cluster my data and ...
Skobo Do's user avatar
-1 votes
1 answer
67 views

how to decide the optimum model?

I have split the database available into 70% training, 15% validation, and 15% test, using holdout validation. I have trained the model and got the following results: training accuracy 100%, ...
user50778's user avatar
0 votes
0 answers
57 views

What are the standard ways to measure the quality of a set of numerical predictions that include uncertainties?

I have a radial basis function that supplies uncertainties (standard deviations) with its predictions, which are numerical values. This function is computed for a particular point by computing its ...
PJ7's user avatar
  • 101
0 votes
0 answers
221 views

Why would my neural network have either an accuracy of 90% or 10% on the validation data, given a random initialization?

I'm making a custom neural network framework (in C++, if that is of any help). When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or ...
Ilknur Mustafa's user avatar
10 votes
3 answers
4k views

Should I choose a model with the smallest loss or highest accuracy?

I have two Machine Learning models (I use LSTM) that have a different result on the validation set (~100 samples data): Model A: Accuracy: ~91%, Loss: ~0.01 Model B: Accuracy: ~83%, Loss: ~0.003 The ...
malioboro's user avatar
  • 2,819
2 votes
0 answers
139 views

Which evaluation metrics should be used in training, validation and testing of a model?

Which specific performance evaluation metrics are used in training, validation, and testing, and why? I am thinking error metrics (RMSE, MAE, MSE) are used in validation, and testing should use a wide ...
user9645302's user avatar