1
$\begingroup$

I am trying to understand how weights are actually gotten. What is generating them in a neural network? What is the algorithm that gives them certain values?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
5
$\begingroup$

Typically, weights are randomly initialized. Then, as the model is optimized for its given task, those weights are steadily made "better" as determined by the network's loss function. This is also referred to as "training" the neural network.

By far the most popular way of updating weights in a neural net is the backpropagation algorithm, most simply with stochastic gradient descent (SGD). Essentially, the algorithm determines how much each individual weight contributed to the network's loss. It then updates that weight in the direction that would reduce the loss.

I recommend going through Michael Nielsen's online book to learn the basics.

Improve this answer
$\endgroup$
0
0
$\begingroup$

I agree with @PhilipRaeisghasem, in most architectures, weights are initialized in a random manner. However, some research papers suggest applying a random normal distribution initialization to the weights in the case of Convolutional Neural Networks (for computer vision).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Absolutely. I didn't think getting into specific initialization distributions would be super beneficial for someone asking such a basic question. But you're right. $\endgroup$
    – Philip Raeisghasem
    Commented Mar 17, 2019 at 23:18
  • $\begingroup$ I absolutely overthought that question. Thanks for replying. $\endgroup$
    – iustin
    Commented Mar 18, 2019 at 21:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .