6 votes
Accepted

Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For ...
user avatar
  • 34.5k
5 votes

Can prior knowledge be encoded in deep neural networks?

Neural nets incorporate prior knowledge. This can be done in two ways: the first (most frequent and more robust) is in data augmentation. For example in convolutional networks, if we know that the "...
user avatar
  • 176
4 votes

Can prior knowledge be encoded in deep neural networks?

Yes, we can do it in a deep learner. For example, suppose we have an input vector likes $(a, b)$ and from prior knowledge, we know $a^2 + b^2$ is important too. Hence, we can add this value to the ...
user avatar
  • 1,663
3 votes

Can prior knowledge be encoded in deep neural networks?

To add to the Foivos's answer, Convolutional Neural Networks are shift-invariant. Fukushima introduced this to his Neocognitron. There is a trail to introduce scale-invariance to CNN. https://arxiv....
user avatar
1 vote

What can be an example for the prior knowledge used in Deep Learning systems?

I would distinguish at least 2 cases when it comes to a generic expression like prior knowledge: generic extra information provide to a model, really close if not the same as feature engineering. ...
user avatar
1 vote

Is the inductive bias always a useful bias for generalisation?

The inductive bias is the prior knowledge that you incorporate in the learning process that biases the learning algorithm to choose from a specific set of functions [1]. For example, if you choose the ...
user avatar
  • 34.5k
1 vote

Is the inductive bias always a useful bias for generalisation?

Is it true that a bias is said to be inductive iff it is useful in generalising the data? Or does inductive bias can also refer to the assumptions that may cause a decrease in performance? Tom M. ...
user avatar
1 vote

Can prior knowledge be encoded in deep neural networks?

It kinda depends on how exactly you define knowledge, and what you believe about what the weights in a trained NN model really represent. But to answer this question in the most straightforward ...
user avatar
  • 3,697

Only top scored, non community-wiki answers of a minimum length are eligible