Questions tagged [bias]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
64 views

In multilayer perceptron, to update the Bias value, do we use the neuron's delta, or do we use the sum of all deltas?

In multilayer perceptron, to update the Bias value, do we use the neuron's delta, or do we use the sum of all deltas? I ask this because I once saw a video on YouTube where the formula for updating ...
will The J's user avatar
0 votes
0 answers
59 views

Why are thresholds in NN so important?

I am working on a paper on the approximation properties of NN. One part of this topic is of course the universal approximation theorem (UAT), which gets discussed in various papers with different ...
Lopsio's user avatar
  • 1
0 votes
0 answers
87 views

How can a bias in a large language model like GPT-4 depend on the languages used for interaction?

Large language models can have biases that depend on cultural biases present in the information they trained on. Some models, including GPT-4, are trained on input data in multiple languages. Some ...
Volker Siegel's user avatar
0 votes
1 answer
205 views

Is there any evidence that the bias terms thelp in the attention mechanism of the transformers?

In the original transformer paper, the attention mechanism uses parameter matrices, but no bias terms. However, in more recent implementations I see people often using a bias term when computing "...
Tarvoc's user avatar
  • 1
0 votes
1 answer
212 views

Confusion about bias in McCulloch-Pitts neurons

I just have a quick question, maybe I am too nit picky here. We recently had an introductory lecture to AI in university and the professor talked about McCulloch-Pitts neurons, e.g. activation as soon ...
DerOeko's user avatar
  • 13
0 votes
1 answer
577 views

Is there a way to freeze training for weights, but not biases in PyTorch? [closed]

I'm constructing a neural network where the weights of my first hidden layer (connected to the input) are all 1 (identity matrix), but the biases are variable. Is there a way to "freeze" any ...
Nova's user avatar
  • 133
2 votes
3 answers
76 views

Why my classification results are correlated with the proportionality of my data?

I'm facing a problem. I'm working on mixed data model with NN (MLP & Word Embedding). My results are not pretty good. And I observed that the proportionality of my data are corelated with my ...
Alexandre Juan's user avatar
1 vote
0 answers
25 views

Does the number of biases in a Convolutional layer scale with the number of images?

I am convolving 32 grey scale images of size 28 x28 with 16 filters of size 5x5. Which of the following is the correct way to add biases to the convolution operation output? add 1 scalar value bias ...
The Ace's user avatar
  • 11
1 vote
1 answer
742 views

Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression

Below are the two tensors ...
ZKS's user avatar
  • 121
0 votes
1 answer
180 views

Bias equal to 1 and neuron output equal to -1 in neural networks

I have read that bias in neural networks is used to avoid situation in which output of neuron is equal to 0. But what if the same output is equal to -1 and we add 1 to it? Isn't it the same issue as ...
bridgemnc's user avatar
0 votes
1 answer
51 views

Can we change bias and control the output of neural network?

I have read the use of Targeted Adversarial Attacks for making the model perform better. But can we change the bias of the neural networks and control the outcome of the network rather than changing ...
harsha's user avatar
  • 101
0 votes
0 answers
420 views

How is the bias added after the convolution in a CNN?

I'm having trouble understanding how bias is added to the feature extraction convolution. I've seen people either refer to the bias as a single number that changes per filter or the whole matrix that ...
kranj doo's user avatar
2 votes
0 answers
51 views

Why do neural networks have bias units?

Why do neural networks have bias units? Why is it sometimes okay to opt them out?
Dinamo's user avatar
  • 21