Questions tagged [validation-loss]

For questions about the validation loss, i.e. the loss of a model (e.g. a neural network) computed on the validation data (or dataset), which is different from the testing/test data (i.e. the data that you use after having trained your model(s) and selected the best model according to the validation loss/performance).

Filter by
Sorted by
Tagged with
0 votes
0 answers
35 views

Eval loss when fine-tuning in an unsupervised way/pretraining?

I'm fine-tuning the base Mixtral 8x7B model (4-bit quantized) with Lora on my own data, following these guidelines: https://www.stochastic.ai/blog/xfinance-vs-bloomberg-gpt I'm first fine-tuning it in ...
Jon Flynn's user avatar
  • 101
0 votes
1 answer
34 views

Validation loss is always lower than training loss whatever i try

I've been training several types of MLPs with different optimisers and tuned them with keras's hyperband tuner. All of them follow this cone architecture: All the networks were trained on the same ...
Roger Smith's user avatar
1 vote
1 answer
145 views

Fluctuations in loss during in epoch evaluation of GRU

I am training a one-layer unidirectional vanilla GRU on a next item prediction task with regard to the last 10 interacted items. In my original experiment, where I trained on approx. 5.5M samples and ...
PatrickSVM's user avatar
0 votes
1 answer
58 views

What are possible reasons for the validation loss increasing with more data?

I trained a neural network on an NLP problem and compared the loss and BLEU score on the validation data with the same training parameters in two scenarios: a) when I trained on 25% of the data, b) ...
postnubilaphoebus's user avatar
0 votes
1 answer
94 views

What would be a good cost function based on both saliency-maps and labels?

I have a number of input samples where: every input sample has both a label and a reference-map. This reference-map gives a score to each location of an input sample. The score defines how much this ...
Wtt's user avatar
  • 1
0 votes
0 answers
104 views

What can cause massive instability in validation loss?

I'm working with very weird data that is apparently very hard to fit. And I've noticed a very strange phenomenon where it can go from roughly 0.0176 validation MSE to 1534863.6250 validation MSE in ...
profPlum's user avatar
  • 424
0 votes
1 answer
318 views

Is it okay to calculate the validation loss over batches instead of the whole validation set for speed purposes?

I have about 2000 items in my validation set, would it be reasonable to calculate the loss/error after each epoch on just a subset instead of the whole set, if calculating the whole dataset is very ...
Ilknur Mustafa's user avatar
1 vote
0 answers
27 views

React on train-validation curve after trening

I have a regression task that I tray to solve with AI. I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance) I understand basic ...
Marko Zadravec's user avatar