# Questions tagged [backpropagation]

For questions about the back-propagation (aka "backprop", and often abbreviated as "BP") algorithm, which is used to compute the gradient of the objective function (e.g. the mean squared error) with respect to the parameters (or weights) of the neural network, when trained with gradient descent.

179 questions
Filter by
Sorted by
Tagged with
125 views

### Data prepared to linear regression. Can I use it with backpropagation?

I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the Java Neural Network Simulator, JavaNNS, program. In one practice I have to build a neural network to use ...
107 views

### Is there any research on neural networks with multiple outputs for hierarchical label classification?

I had this idea of training for example a CNN on images, and having output branches at several of its intermediate layers. The early layers' output branch might then predict high-level class of ...
112 views

### What are some concrete steps to deal with the vanishing gradient problem?

I am training an ANN for classification between 3 classes. The ANN has an input layer, one hidden layer and a 3 node output layer. The problem I am facing is that the output being produced by the 3 ...
282 views

### What is the proof behind the gradient of a curve being proportional to the distance between the two co-ordinates in the x-axis?

In the [delta rule] the equation to adjust the weight with respect to error is $$w_{(n+1)}=w_{(n)}-\alpha \times \frac{\partial E}{\partial w}$$ *where $\alpha$ is the learning rate and $E$ is the ...
466 views

### What is the relation between back-propagation and reinforcement learning?

What is the relation between back-propagation and reinforcement learning?
187 views

### Why does ReLU (and other non linearities) work?

Can someone please point me to where I can read up on why non linearities that can produce values larger than 1 or smaller than 0 work. My understanding is that neurons can only produce values between ...
4k views

### How to combine backpropagation in neural nets and reinforcement learning?

As I am trying to make an AI with reinforcement learning, I have found out and implemented a lot of things such as both these topics (NNs and RL) separately. But when trying to combine them, I have ...
104 views

### When do you back-propagate errors through a neural network when using TD($\lambda$)?

I have a neural network that I'm want to use to self-play Connect Four. The neural network receives the board state and is to provide an estimate of the state's value. I would then, for each move, ...
141 views

### How to design 4D Deep Recurrent Neural Networks using Tensorflow?

I want to design a simple model that predicts the movement of coordinates with RNNs. In a typical three-dimensional LSTM model, one feature is encoded as one hot encoding, and the ...
760 views

### Forecasting and predict using matlab Artificial Neural Network [closed]

I selected the below data set for forecast and predict using artificial neural network as my final year project. https://archive.ics.uci.edu/ml/datasets/Bank+Marketing. I normalized the data set and ...
227 views

### Finding an optimum back propagation algorithm

I recently started working on very simple machine learning codes in Python and I came across a big problem: teaching the system to improve on its guesses. So this is what the code is about: I will ...
47 views

### Recommendations on which architecture to use to guess appointment

I'm currently developping an application which allows psychologists to manage their schedule and budget. As a proof of concept, I would like to create an intelligent appointment service. There can be ...
544 views

### What makes learned feature detectors specialize in CNN?

It has been shown that it is possible to use unsupervised learning techniques to produce good feature detectors in CNNs. I can't understand what drives specialization of those feature detectors. In ...
172 views

### What would an implementation of this Neural Network look like?

I'm relatively new to neural networks and was wondering what an implementation of this paper would look like. More specifically, how are the correct values of Kp, Ki, and Kd determined at run time so ...
706 views

### How do I know if my backpropagation is implemented correctly?

I'm working on implementation of the backpropagation algorithm for a simple neural network which predicts a probability of survival (1 or 0) and I can't get it above 80% no matter how much I try to ...
2k views

### Is the mean-squared error always convex in the context of neural networks?

Multiple resources I referred to mention that MSE is great because it's convex. But I don't get how, especially in the context of neural networks. Let's say we have the following: $X$: training ...
153 views

### Can a neural network learn to avoid wrong decisions using backpropagation?

I studied the articles on Neural Networks and Deep Learning from Michael Nielsen and developed a simple neural network based on his examples. I understand how backpropagation works and I already ...
70 views

### Can a second network take as input the weights of a first network and help training the first network?

I understand that as a network learns about an output with regards to an input, weights are updated according to how wrong the guess was for that node. So, over time, the weights move in the "...
340 views

### What is the order of execution of steps in back propagation algorithm in a neural network?

I am a machine learning newbie. I am trying to understand backpropagation algorithm. I have a training dataset of 60 instances/records. So what is the correct order of the process: Forward pass of ...
709 views

### Are Dreams a Form of Backpropagation?

Humans often dream of random events that occurred during the day. Could the reason for this be that our brains are backpropagating errors while we sleep, and we see the result of these ...
2k views

### How to test if my implementation of back propagation neural Network is correct

I am working on an implementation of the back propagation algorithm. What I have implemented so far seems working but I can't be sure that the algorithm is well implemented, here is what I have ...
241 views

### Why doesnt my Neural Network work?

I Build this NN in c++. I reviewed it since 3 days. I checked every line 100 times, but I cant find my error. If someone can please help me find the Bugs: 1. The output is garbage 2. The weights go ...
102 views

### How to perform back-propagation in Decoupled Neural Interfaces?

I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (...
313 views

### What kind of algorithm is the Levenberg–Marquardt algorithm?

Is a Levenberg–Marquardt algorithm a type of back-propagation algorithm or is it a different category of algorithm? Wikipedia says that it is a curve fitting algorithm. How is a curve fitting ...
433 views

### Are these two versions of back-propagation equivalent?

Just for fun, I am trying to develop a neural network. Now, for backpropagation I saw two techniques. The first one is used here and in many other places too. What it does is: It computes the ...
142 views

### How do evolutionary algorithms have advantages over the conventional backpropagation methods?

How does employing evolutionary algorithms to design and train artificial neural networks have advantages over using the conventional backpropagation algorithms?
3k views

### How to avoid falling into the “local minima” trap?

How do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network? Are there any methods which help me avoid it?