Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.
12

"Backprop" is the same as "backpropagation": it's just a shorter way to say it. It is sometimes abbreviated as "BP".


8

I haven't seen an answer from a trusted source, but I'll try to answer this myself, with a simple example (with my current knowledge). In general, note that training a MLP using back-propagation is usually implemented with matrices. Time complexity of matrix multiplication The time complexity of matrix multiplication for $M_{ij} * M_{jk}$ is simply $\...


8

In reverse order to how you asked: all units in a layer become equal since initially the errors due to all of them are the same and thus we train them to be equal This actually happens if you initialise the weights equally (e.g. all zero). Gradients in that case are the same to each neuron in the same layer, and everything changes in lockstep. A neural ...


7

'Backprop' is short for 'backpropagation of error' in order to avoid confusion when using backpropagation term. Basically backpropagation refers to the method for computing the gradient of the case-wise error function with respect to the weights for a feedforward networkWerbos. And backprop refers to a training method that uses backpropagation to compute ...


7

Yes this is done routinely. For example this is how the YOLO object detection and classifier system works, to give a real-world for example. In YOLO, the "non-object" classification is "background" i.e. any image segment that doesn't contain one of the types of object we are interested in. In general, you can add an "other" class to any classifier, provided ...


6

Unlike backpropagation, evolutionary algorithms do not require the objective function to be differential with respect to the parameters you aim to optimize. As a result, you can optimize "more things" in the network, such as activation functions or number of layers, which wouldn't be possible in the standard backpropagation. Another advantage is that by ...


6

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include: Probabalistically accepting worse solutions in the hope that this will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). Maintaining a list of recently-encountered states (or attributes ...


6

Backpropagation is a subroutine often used when training Artificial Neural Networks with a Gradient Descent learning algorithm. Gradient Descent requires computation of the error gradient, i.e. derivatives, of a cost function with respect to the network parameters. BP allows you to find this gradient a lot faster than using naive methods. Reinforcement ...


6

Backpropagation is used to update the weights in a neural network. A possible implementation is to map a (state, action)-pair to a Q-value. Gradient descent can be used as an optimizer to learn a policy for an agent. Action selection The action that yields the highest Q-value is chosen in a particular state. A neural network can be designed in many ...


5

When you are training a neural network, you use an algorithm called back propagation. This algorithm uses partial derivatives to determine the optimal values for weights. Partial derivatives are a calculus based method which tell you how far you need to adjust the weights in order to get to an optimum value. However, when you have neural networks with many ...


5

The two examples present essentially the same operation: In both cases the network is trained with gradient descent using the backpropagated squared error computed at the output. Both examples use the logistic function for node activation (the derivative of the logistic function s is s(1 - s). This derivative is obviously very easy to compute, and this is ...


5

Further to Franck's answer, there may be better optima (even global optima) that exist in the opposite direction to the gradient (which may be in the direction of some local optima). Evolutionary algorithms have scope to search the surrounding area, while backpropagation will always move in the direction of the gradient. With no guarantee (due to their ...


5

You can run gradient descent without back propagation, in some cases: Simple structures such as linear or logistic regression, where the gradients can be calculated directly from the inputs and cost function value. In "black box" gradient-based learning algorithms where you don't know how (or don't want to) calculate gradient analytically, so you choose to ...


4

For future reference, I will merely point you to a technique you can implement to test the correctness or lack thereof, of your backpropagation implementation. Ps: don't feel too bad for having gotten it slightly wrong, "backpropagation is notoriously difficult to implement" - source :). In fact, there is a technique called "Gradient checking" meant ...


4

Actually the implementation was correct, The source of the problem that causes a big error and really slow learning was the architecture of the neural network it self, the ANN has 7 hidden layers which causes the vanishing gradient problem. When I have decreased the ANN layers to 3 the cost of error was widely reduced besides of that the learning process ...


4

Gradient descent (GD) is an optimisation algorithm, that is, it is used to find a (local) minimum of a multi-variable and differentiable function $f$. GD is an iterative and numerical optimisation algorithm. It is iterative because it proceeds in iterations. It is numerical because it is not an algorithm which produces an exact solution, due to numerical ...


3

You should read up on these papers: Deep Q-Networks Asynchronous Deep Reinforcement Learning Both by DeepMind, they achieved super-human results on video-games and other tasks. They describe the algorithms quite well. It is not as simple as the previous answer, which won't converge to a policy in complex environments.


3

I believe the best way to do this is using numerical gradient. To understand the concept, we need to look the definition of derivatives using limits: It means that, when you don't know how to derive some formula (or you just don't want to), you can approximate it by computing the output for a small change in input, subtract from the original result (no ...


3

There is not single answer to the vanishing gradient problem. However, there a few things that can help. As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs. Next, careful choice of weight initialization can help avoid saturation, as well. See ...


3

This is too broad a topic to answer directly. If you are at the beginner stage with neural networks, you will need to learn some basic theory of the maths of neural networks, before the code will make sense. Although it is possible to write neural network code with only a vague understanding of what is going on, it is not a great way to learn for the ...


3

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ...


3

Let's define your problem from another point of view. Let's say that in this RL problem you have two agents (agent1 and agent2) that compete with each other in order to accomplish their own goal, i.e., wining connect4 game. Therefore, we could say that from agent1's point of view, he is player1 and the player2 is agent2. The same way, from agent2's point of ...


3

I think what you mean to ask is how can differentiation occur when there's no obvious neural network function to differentiate? Don't worry - lots of people get confused about this, because it seems like an obvious hole in the puzzle. As mentioned by @AtillaOzgur, neural networks use partial differentiation through backpropagation. First, take the output ...


2

Yes, as Franck has rightly put, "backprop" means backpropogation, which is frequently used in the domain of neural networks for error optimization. For a detailed explanation, I would point out this tutorial on the concept of backpropogation by a very good book of Michael Nielsen.


2

Multilayer Perceptron (MLP) can theoretically approximate any bounded, continuous function. There's no guarantee for a discontinuous function. There are plenty of important discontinuous functions, like, say, the prime counting function. The prime counting function pi(n) is simply equal to the number of primes less than or equal to n. It has a discontinuity ...


2

While I'm not familiar with any explicit statements regarding what a Multilayer Perceptron (MLP) cannot learn, I can provide some further detail on the positive statements you made about MLP capabilities: A MLP with a single hidden layer is capable of what is commonly termed 'Universal Function Approximation', i.e. it can approximate any bounded continuous ...


2

One thing that you should know is that the sigmoid function limits the output to a value between 0 and 1, which means that using a lot of hidden layers will lead quickly to a Vanishing Gradient. Try to use relu activation function, it has the property to output all the information it gets from the previous layer.


2

A little search on Google answers your question. XOR input space is not linearly separable. It means that you cannot separate the input points in a 2D space into 1 area and 0 area by simply drawing a line between them. It requires at least 2 lines to separate the XOR input space and consequently 2 output nodes (used as classifiers rather than regression). ...


2

Don’t think about it as the being proportional to something. Think about it this way: I’m now at . Where do I want to be at Time step so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of If increases as I increase (that is, if , then obviously, I would want to move a little bit to ...


2

Christopher Olah's blog post describes it better that I ever could. Basically, most data we come across can't be separated with a single line, but with some kind of curve. Non-linearities allow us to distort the input space in ways that make the data linearly separable, making classification more accuarate.


Only top voted, non community-wiki answers of a minimum length are eligible