# Questions tagged [backpropagation]

For questions about the back-propagation (aka "backprop", and often abbreviated as "BP") algorithm, which is used to compute the gradient of the objective function (e.g. the mean squared error) with respect to the parameters (or weights) of the neural network, when trained with gradient descent.

215 questions
Filter by
Sorted by
Tagged with
30 views

29 views

### XOR problem with bipolar representation

I am taking a course in Machine Learning and the Professor introduced us to the XOR problem. I understand the XOR problem is not linearly separable and we need to employ Neural Network for this ...
78 views

### How is the error calculated with multiple output neurons in the neural network?

Machine Learning books generally explains that the error calculated for a given sample $i$ is: $e_i = y_i - \hat{y_i}$ Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
28 views

### In LSTMs, how does the additive property enables better balancing of gradient values during backpropagation?

There are two sources that I'm using to to try and understand why LSTMs reduce the likelihood of the vanishing gradient problem associated with RNNs. Both of these sources mention the reason LSTMs are ...
31 views

### Linear output layer back propagation

So I'm stack to something that it's probably very easy but I can't get my head around it. I'm building a Neural Network that will consist of many layers with non-linear activation functions (probably ...
68 views

### Computation of initial adjoint for NODE

I'm reading the paper Neural Ordinary Differential Equations and I have a simple question about adjoint method. When we train NODE, it uses a blackbox ODESolver to compute gradients through model ...
129 views

### How does back-propagation through time work for optimizing the weights of a bidirectional RNN?

I am aware that back-propagation through time is used for training the recurrent neural network. But I am not able to understand how this happens for the bi-directional versions of the recurrent ...
83 views

### What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?

Let us assume we have a GRU network containing $H$ layers to process a training dataset with $K$ tuples, $I$ features, and $H_i$ nodes in each layer. I have a pretty basic idea how the complexity of ...
56 views

### How are weight matrices in attention trained?

I have been looking into transformers lately and been reading tons of tutorials. All of them address the intuition behind attention, which I understand, but they treat training the weight matrices for ...
I am trying to comprehend how the Gradient Descent works. I understand we have a cost function which is defined in terms of the following parameters, $J(𝑤_{1},𝑤_{2},.... , w_{n}, b)$ the derivative ...
My doubt is like this : Suppose we have an MLP. In an MLP, as per the backprop algorithm (back-propagation algorithm), the correction applied to each weight is :  w_{ij} := -\eta\frac{\partial E}{\...