2
$\begingroup$

I noticed that the TensorFlow library includes a use_bias parameter for the Dense layer, which is set to True by default, but allows you to disable it. At first glance, it seems unfavorable to turn off the biases, as this may negatively affect data fitting and prediction.

What is the purpose of layers without biases?

$\endgroup$
4
  • $\begingroup$ Before anything, having option is not bad. Besides I think some applications do not need bias. For example if if I'm not wrong, in CBOW (Continuous Bag Of Words for word embedding) you don't need bias. $\endgroup$
    – amin
    Dec 6 '20 at 8:56
  • $\begingroup$ @amin You say "Besides I think some applications do not need bias.", but I think the question is really: "Why do some applications not use/need bias". If you don't have the answer to this question, besides providing the usual explanation of why the bias can be useful, then I think you should upvote this question/post, so that maybe it attracts someone that knows the answer. Personally, I don't know the answer of when you would strictly not need the bias (and thus when you can disable it), so it would be nice to have some guidance on when to disable it (because not really necessary). $\endgroup$
    – nbro
    Dec 6 '20 at 12:17
  • $\begingroup$ @nbro, is this the right stackexchange to ask questions like this, or it is more datascience? Where is the line drawn? $\endgroup$
    – mark mark
    Dec 9 '20 at 22:52
  • $\begingroup$ @markmark This is a theoretical ML question, so it is perfectly suitable for AI SE, and I think it's a question that other people could also have, so I think it's useful. Unfortunately, it has not received too much attention, but this is more or less normal (especially, in the last months): sadly, there aren't many people contributing to the site as much as I would like. In any case, I think this question would also be on-topic on Data Science SE and Cross Validated SE. $\endgroup$
    – nbro
    Dec 9 '20 at 23:00
1
$\begingroup$

Bias is one of the hyperparameters in neural networks, which let you shift activation function. Disabling bias means setting bias to be zero.

Even though, in many cases, bias is a big help for successful learning, in some cases, you may want to add an extra constraint to your neural network in finding the objective function. For example, in the paper below, a zero-bias layer as the last layer helps increase the output's interpretability.

https://ieeexplore.ieee.org/abstract/document/9173537

$\endgroup$
1
  • $\begingroup$ It would be nice if you could describe in which sense no bias makes the output more interpretable (according to that paper). $\endgroup$
    – nbro
    Dec 6 '20 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.