Questions tagged [backpropagation]

For questions related to the technique of backpropagation, whereby the loss, error, or correction signal calculated at the output of an artificial network output is fed back to the parameters in each layer of the network until the network's behavior converges to a training state within the required accuracy and reliability.

Filter by
Sorted by
Tagged with
0
votes
0answers
25 views

How to find the derivative of a dynamic neuron model, which depends on previous states of the neuron?

This is the equation where n denotes the current state, (n-1) denotes the state in the previous step etc. And to do back-propagation I need to find partial derivatives over each of the variables. For ...
0
votes
0answers
22 views

Finding the energy function given update rule of a single layer non-linear neural network

Consider the network with N neurons, each of which takes a $2 \times k$ input specified by the tuple $(\vec c_t, \vec \theta_t)$ to produce output $\vec{R}_t$ through an update rule on the pairwise ...
0
votes
0answers
55 views

Backpropagation implementation with Java

I've been trying to implement a Multilayer Perceptron Network using java language with the ultimate goal of creating and teaching a neural network to recognize handwritten digits. Pretty simple and ...
0
votes
0answers
41 views

Why is it that having a duplicate in features set makes training to work bad

I'm defining a deep network to emulate a multitarget regression. When I costruct my training set, I take information from a graph; without going into too much detail, it could happen that I take 2 ...
0
votes
0answers
67 views

Backward pass of CNN like Resnet: how to manually compute flops during backprop?

I've been trying to figure out how to compute the number of Flops in backward pass of ResNet. For forward pass, it seems straightforward: apply the conv filters to the input for each layer. But how ...
0
votes
2answers
59 views

Backpropagation of neural nets with shared weight

I am trying to understand the mathematics behind the forward and backward propagation of neural nets. To make myself more comfortable, I am testing myself with an arbitrarily chosen neural network. ...
0
votes
0answers
18 views

Why do code implementations average the loss over a batch instead of finding the expected sample of that batch (using sampling probabilities)

Usually, our training objective over a batch is written in terms of the expected value of a sample in that batch such as $objective = E_{x \sim data} * log(P(x))$ But in the code implementations, ...
0
votes
1answer
67 views

Is my backpropagation code correct? [closed]

I am trying to implement the back-propagation algorithm for the following neural network. ...
0
votes
0answers
57 views

Derivation of regularized cost function w.r.t activation and bias

In regularzied cost function a L2 regularization cost has been added. Here we have already calculated cross entropy cost w.r.t $A, W$. As mentioned in the regularization notebook (see below) in ...
0
votes
0answers
13 views

How to perform PGD on a pretrained CNN?

I have a pretrained CNN model using the keras library. I now need to perform a Projected Gradient Descent (PGD) to develop some adversarial examples. To do this, I will need to perform a gradient ...
0
votes
1answer
22 views

Positive bias causes the calculation of incorrect gradients

I have a data set with a positive bias (an image, where the values range from 0 to 1), that seems to be causing my network to calculate incorrect gradients. If I just use the raw image as input, of ...
0
votes
0answers
28 views

Backpropagation and PID

In backpropagation, the update rule for the weights is based on the derivative of some loss function. This is similar to the "proportional" aspect of a PID loop controller, where some control variable ...
0
votes
1answer
35 views

Backpropagation: Chain Rule to the Third Last Layer

I'm trying to solve dLoss/dW1. The network is as in picture below with identity activation at all neurons: Solving dLoss/dW7 is simple as there's only 1 way to output: $Delta = Out-Y$ $Loss = abs(...
0
votes
0answers
50 views

Confused about NeuralODE

I am a bit confused about NeuralODE and I want to make sure that what I understood so far is correct. Assume we have (for simplicity) 2 data points $z_0$ measured at $t_0$ and $z_1$ measured at $t_1$...
0
votes
2answers
368 views

How should the values of the filters of a CNN change?

I wrote a convolutional neural network for the MNIST dataset with Numpy from scratch. I am currently trying to understand every part and calculation. But one thing I noticed was the "just positive" ...
0
votes
1answer
446 views

Back propagation on Flatten Layer in CNN

I am making a NN library without any other external NN lib and is implementing the Flatten layer. I know the forward implementation of flatten layer but is the backward just reshaping it or not? If ...
0
votes
0answers
170 views

How to back-propagate illegal actions for policy gradient learning

When training a AI RL agent to play a game there'll be situations where the AI cannot perform certain actions lest they violate the game rules. That's easy to handle, and I can set illegal actions to ...
0
votes
1answer
80 views

Back propagation on matrix of weights

I am trying to implement a Neural Network for binary classification using python and numpy only. My network structure is as follows: input features: 2 [1X2] matrix Hidden layer1: 5 neurons [2X5] ...

1 2 3
4