Questions tagged [weights]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
45 views

When would bias regularisation and activation regularisation be necessary?

For Keras on TensorFlow, a layer class constructor comes with these: kernel_regularizer=... bias_regularizer=... ...
1
vote
0answers
31 views

LSTM - MAPE Loss Function gives Better Results when Data is De-Scaled before Loss Calculation

I am building an LSTM for predicting a price chart. MAPE resulted in the best loss function compared to ...
2
votes
1answer
53 views

Is better to spend parameters on weights or bias?

If a neural network has a limited number of neuron parameters to find, -let's say only 1000 parameters-, it is generally better to spend the parameters on weights or neuron bias? For example, if each ...
0
votes
2answers
58 views

Backpropagation of neural nets with shared weight

I am trying to understand the mathematics behind the forward and backward propagation of neural nets. To make myself more comfortable, I am testing myself with an arbitrarily chosen neural network. ...
1
vote
2answers
72 views

How are non-linear surfaces formed in the training of a neural network?

Desperate trying to understand something for couple of weeks. All those questions are actually one big question.Please help me. Time-codes and screens in my question refer to this great(IMHO) 3d ...
1
vote
0answers
37 views

Is having binary randomized unchanging neural network weights a good idea?

I am creating a neural network to experiment with, and I was wondering: If I have weights randomly initialized to be either 1 or 0 for each neuron, and then I made it so that the weights cannot be ...
2
votes
1answer
25 views

How are weights for weighted x-entropy loss on imbalanced data calculated?

I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the advice of the tensorflow docs ...
1
vote
1answer
29 views

Why I have a different number of terms in word2vec and TFIDF? How I can fix it?

I need multiply the weigths of terms in TFIDF matrix by the word-embeddings of word2vec matrix but I can't do it because each matrix have a different number of terms. I am using the same corpus for ...
1
vote
0answers
28 views

Function to update weights in back-propagation

I am trying to wrap my head around how weights get updated during back propagation. I've been going through a school book and I have the following setup for an ANN with 1 hidden layer, a couple of ...
2
votes
1answer
35 views

How are newer weight initialization techniques better than zero or random initialization?

How do newer weight initialization techniques (He, Xavier, etc) improve results over zero or random initialization of weights in a neural network? Is there any mathematical evidence behind this?
1
vote
0answers
35 views

Interpretability of feature weights from Gaussian process classifier

Suppose I trained a Gaussian process classifier with a linear kernel (using GPML toolbox) and got some feature weights for each input feature. My question is then: Does it/When does it make sense ...
3
votes
1answer
115 views

How does the neural-network know how to tweak weights for a specific neuron?

I know backpropagation uses cost and gradient descent to tweak the weights to minimize the cost. But how does it know which weights to give more weight to in the first place? Is there something inside ...
1
vote
1answer
46 views

Classification with deeplearning : clean start vs continue training

I trained some weights to identify apples and oranges (using YOLOv3). If I want to be able to identify peaches, which approach is usually recommended: Start clean and train the 3 classes. Train ...
1
vote
1answer
43 views

How to plot Loss Landscape with more than 2 weights in the network

For a single neuron with 2 weights, I can plot the loss landscape and it looks like this (OR data, sigmoid activation, MAE loss): But, when the neuron accepts more inputs, which means more than 2 ...