0
$\begingroup$

If we use two loss functions that are in the same "units" (say MAE and RMSE). Should training the same model with these two different loss functions result in plateau happening at a different place?

Say in this particular case our dataset does not have many outliers so there's not much penalizing the outliers.

$\endgroup$

1 Answer 1

0
$\begingroup$

If you train the same model with two different loss functions, such as MAE and RMSE, the model may reach a plateau at different places for each loss function. This is because these loss functions have different sensitivities to errors and are optimized differently during training.

In your specific case, where there are not many outliers in the dataset, the differences in the training dynamics between MAE and RMSE might be less pronounced. However, it is still possible for the models to reach a plateau at different points, because the two loss functions handle errors differently. The model with RMSE may converge slightly later than the one with MAE because it focuses more on reducing larger errors.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .