Questions tagged [weights]
The weights tag has no usage guidance.
14
questions
1
vote
1answer
45 views
When would bias regularisation and activation regularisation be necessary?
For Keras on TensorFlow, a layer class constructor comes with these:
kernel_regularizer=...
bias_regularizer=...
...
1
vote
0answers
31 views
LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation
I am building an LSTM for predicting a price chart. MAPE resulted in the best loss function compared to ...
2
votes
1answer
53 views
Is better to spend parameters on weights or bias?
If a neural network has a limited number of neuron parameters to find, -let's say only 1000 parameters-, it is generally better to spend the parameters on weights or neuron bias?
For example, if each ...
0
votes
2answers
58 views
Backpropagation of neural nets with shared weight
I am trying to understand the mathematics behind the forward and backward propagation of neural nets. To make myself more comfortable, I am testing myself with an arbitrarily chosen neural network. ...
1
vote
2answers
72 views
How are non-linear surfaces formed in the training of a neural network?
Desperate trying to understand something for couple of weeks. All those questions are actually one big question.Please help me. Time-codes and screens in my question refer to this great(IMHO) 3d ...
1
vote
0answers
37 views
Is having binary randomized unchanging neural network weights a good idea?
I am creating a neural network to experiment with, and I was wondering:
If I have weights randomly initialized to be either 1 or 0 for each neuron, and then I made it so that the weights cannot be ...
2
votes
1answer
25 views
How are weights for weighted x-entropy loss on imbalanced data calculated?
I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the advice of the tensorflow docs
...
1
vote
1answer
29 views
Why I have a different number of terms in word2vec and TFIDF? How I can fix it?
I need multiply the weigths of terms in TFIDF matrix by the word-embeddings of word2vec matrix but I can't do it because each matrix have a different number of terms. I am using the same corpus for ...
1
vote
0answers
28 views
Function to update weights in back-propagation
I am trying to wrap my head around how weights get updated during back propagation. I've been going through a school book and I have the following setup for an ANN with 1 hidden layer, a couple of ...
2
votes
1answer
35 views
How are newer weight initialization techniques better than zero or random initialization?
How do newer weight initialization techniques (He, Xavier, etc) improve results over zero or random initialization of weights in a neural network? Is there any mathematical evidence behind this?
1
vote
0answers
35 views
Interpretability of feature weights from Gaussian process classifier
Suppose I trained a Gaussian process classifier with a linear kernel (using GPML toolbox) and got some feature weights for each input feature.
My question is then:
Does it/When does it make sense ...
3
votes
1answer
115 views
How does the neural-network know how to tweak weights for a specific neuron?
I know backpropagation uses cost and gradient descent to tweak the weights to minimize the cost. But how does it know which weights to give more weight to in the first place? Is there something inside ...
1
vote
1answer
46 views
Classification with deeplearning : clean start vs continue training
I trained some weights to identify apples and oranges (using YOLOv3).
If I want to be able to identify peaches, which approach is usually recommended:
Start clean and train the 3 classes.
Train ...
1
vote
1answer
43 views
How to plot Loss Landscape with more than 2 weights in the network
For a single neuron with 2 weights, I can plot the loss landscape and it looks like this (OR data, sigmoid activation, MAE loss):
But, when the neuron accepts more inputs, which means more than 2 ...
