Timeline for Is there a way to ensure that my model is able to recognize an unseen example?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 12, 2021 at 13:05 | history | edited | nbro | CC BY-SA 4.0 |
added 15 characters in body; edited tags
|
| Feb 26, 2020 at 22:14 | answer | added | Alpha | timeline score: 1 | |
| Feb 25, 2020 at 21:40 | comment | added | nbro | To answer your question directly, regularization may help, but it may depend on the regularization that you use. | |
| Feb 25, 2020 at 21:38 | comment | added | nbro | The term regularization is often used ambiguously. For example, in L1 or L2 regularization, the goal is to constrain the parameters to lie in a certain range. In other cases, for example, in the case of dropout, the idea is that you attempt to decorrelate the units of the network (by randomly dropping or killing them during training). In the case of Bayesian neural networks, the regularisation is said to be integrated into the networks because they are based on Bayes rule, so, in a certain way, the uncertainty associated with the learned distributions depends on the available data. | |
| Feb 25, 2020 at 21:29 | comment | added | mdurrant | Would regularizing the model even more than seemed necessary given your available dataset help you to generalize to these more dissimilar, unseen examples? | |
| Feb 25, 2020 at 21:28 | comment | added | mdurrant | So assume the typical train-dev-test paradigm, where all three sets have unique examples but are drawn from the same distribution. With a neural network, you can easily overfit to those particular examples that exist in the training set if you don't regularize the parameters. Regularization allows the model to learn a more generalizable function. But if an unseen example that is not well represented in the train, dev, or test sets is introduced, how can we ensure that the model will be more likely to generalize to this distantly related, new example? | |
| Feb 25, 2020 at 18:32 | comment | added | nbro | Dropout is just one way of introducing a regularization effect. There are others. Dropout should help to avoid overfitting and thus the network shouldn't memorize the training dataset (i.e. approximate the function that the training set represents), but, instead, approximate a different function (not the one represented by the training set). Clearly, this should help to predict something that isn't in the training set, but this will not ensure that a specific unseen example will be predicted correctly. | |
| Feb 25, 2020 at 18:01 | comment | added | mdurrant | I am tempted to conclude that the best way to ensure that the network still performs well on unseen examples is to regularize it as much as possible (Dropout rate of 0.5 at every layer, for example). Do you think this intuition is correct? | |
| Feb 25, 2020 at 1:28 | history | edited | nbro | CC BY-SA 4.0 |
The original title is directly related to the actual question.
|
| Feb 24, 2020 at 21:35 | review | First posts | |||
| Feb 24, 2020 at 23:13 | |||||
| Feb 24, 2020 at 21:31 | history | asked | mdurrant | CC BY-SA 4.0 |