Timeline for What can be an example for the prior knowledge used in Deep Learning systems?

4 events

when toggle format	what		by	license	comment
Oct 8, 2021 at 12:30	comment	added	nbro		Note that I am not saying that BNNs are not a good example where we can introduce prior knowledge. In fact, in BNNs, we can really specify priors for the weights (as priors in the Bayes' rule, which are used to introduce "prior knowledge"), but it just seems that you were referring to the initialization of the weights and other techniques to guide the training of non-Bayesian neural networks. Another thing you may want to point out is the relationship between "prior knowledge" and "inductive bias" (and/or model architecture).
Oct 8, 2021 at 12:29	comment	added	nbro		This answer provides good information, but it mixes concepts, so it may be confusing. For example, you say that the second way to interpret "prior knowledge" is "literal prior probability distributions used to initialize or guide a model during training", but then you give the example of "Bayesian neural networks". It's true that BNNs are based on having distributions over the weights, but in your second point you mention probability distributions to initialize or guide a model during training, which seemed to suggest that you were referring to ways to initialise the weights.
Oct 8, 2021 at 10:25	history	edited	Edoardo Guerriero	CC BY-SA 4.0	deleted 2 characters in body
Oct 8, 2021 at 9:05	history	answered	Edoardo Guerriero	CC BY-SA 4.0