There's a core problem with all of ML which I haven't really seen made explicit: the issue is every model needs to have an assumption on the structure of the data you learn and this assumption needs to be expressed inside the structure of your ML model.

Implicit in all of successful DL is heuristics in regards to restricting the structure of networks to be consistent with image data(ex hierarchical features, translation invariance, etc).

What I am curious about is if anyone has attempted to explicitly encode these types of priors into DL models?

  • $\begingroup$ Can you clarify what you mean by "explicitly" in "explicitly encode these types of priors"? What exactly would you have in mind? Do you mean using some kind of prior probability distribution? If yes, how exactly would you think to do that? If not, explain what you mean. $\endgroup$
    – nbro
    Dec 13 '21 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.