Return to Answer

This derivative is used when calculating the error of your machine learning algorithm during gradient based minimization methods. Read below for more info.

When performing supervised classification (with (x,y) data to train) you begin with the error function E= (f(x; theta)-y)^2 (sum over all training instances (x_i,y_i) for overall error), where f is your neural network, linear regression,...method of interest and theta is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). Note f(x) outputs value of interest which should be y. And E measures how far off it is in prediction.

So to train your classifier, you optimize E with something like gradient descent. Thus when the derivative (wrt theta) is zero, that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.

Note the f here is not the same as an activation function, as a neural network must perform a special kind of gradient descent called backpropagation. But overall the idea is similar.

So when you take partial derivative of the error expression above w.r.t. theta, where does it equal? You should note the activation functions derivative is involved which is how it’s used to measure error so to say.

This derivative is used when calculating the error of your machine learning algorithm during gradient based minimization methods. Read below for more info.

When performing supervised classification (with X, Y data vectors of inputs and outcome data to train with) you begin with the error function

E(X, Y; θ)= ∑_i (ƒ(x_i; θ)-y_i)²

for total error over all data instances i, where f is your neural network, linear regression,...method of interest and θ is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). To be explicit, ƒ(x_i; θ); outputs value of interest which should be y_i. And E measures how far off it is in prediction.

So to train your classifier, you optimize E with something like gradient descent. Thus when ∂E/∂θ = 0 (for a particular θ), that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.

Note the ƒ here is not the same as an activation function, as a neural network is defined differently than in linear regression, etc. and must perform a special kind of gradient descent called backpropagation.

So when you take ∂E/∂θ, what does it equal for a neural net? You should note the activation functions derivative is involved which is how it’s used to measure error so to say.

This derivative is used when calculating the error of your machine learning algorithm during gradient based minimization methods. Read below for more info.

When performing supervised classification (with (x,y) data to train) you begin with the error function E= (f(x; theta)-y)^2 (sum over all training instances (x_i,y_i) for overall error), where f is your neural network, linear regression,...method of interest and theta is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). Note f(x) outputs value of interest which should be y. And E measures how far off it is in prediction.

So to train your classifier, you optimize E with something like gradient descent. Thus when the derivative (wrt theta) is zero, that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.

Note the f here is not the same as an activation function, as a neural network must perform a special kind of gradient descent called backpropagation. But overall the idea is similar.

So when you take partial derivative of the error expression above w.r.t. theta, where does it equal? You should note the activation functions derivative is involved which is how it’s used to measure error so to say.

Will typeset this properly when I get the chance.

This derivative is used when calculating the error of your machine learning algorithm during gradient based minimization methods. Read below for more info.

When performing supervised classification (with (x,y) data to train) you begin with the error function E= (f(x; theta)-y)^2 (sum over all training instances (x_i,y_i) for overall error), where f is your neural network, linear regression,...method of interest and theta is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). Note f(x) outputs value of interest which should be y. And E measures how far off it is in prediction.

So to train your classifier, you optimize E with something like gradient descent. Thus when the derivative (wrt theta) is zero, that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.

Note the f here is not the same as an activation function, as a neural network must perform a special kind of gradient descent called backpropagation. But overall the idea is similar.

So when you take partial derivative of the error expression above w.r.t. theta, where does it equal? You should note the activation functions derivative is involved which is how it’s used to measure error so to say.

Although I’m not familiar with it’s application above, when performing supervised classification (with (x,y) data to train) you begin with the error function E= (f(x; theta)-y)^2 (sum over all training instances (x_i,y_i) for overall error), where f is your neural network, linear regression,...method of interest and theta is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). Note f(x) outputs value of interest which should be y. And E measures how far off it is in prediction.

So to train your classifier, you optimize E with something like gradient descent. Thus when the derivative (wrt theta) is zero, that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.

Note the f here is not the same as an activation function, as a neural network must perform a special kind of gradient descent called backpropagation. But overall the idea is similar.

So when you take partial derivative of the error expression above w.r.t. theta, where does it equal? You should note the activation functions derivative is involved which is how it’s used to measure error so to say.

Will typeset this properly when I get the chance.

This derivative is used when calculating the error of your machine learning algorithm during gradient based minimization methods. Read below for more info.

When performing supervised classification (with (x,y) data to train) you begin with the error function E= (f(x; theta)-y)^2 (sum over all training instances (x_i,y_i) for overall error), where f is your neural network, linear regression,...method of interest and theta is the set of weights. The goal here is to find weights that minimize your error when predicting training data (y) (which ideally generalizes to new data as well). Note f(x) outputs value of interest which should be y. And E measures how far off it is in prediction.

So to train your classifier, you optimize E with something like gradient descent. Thus when the derivative (wrt theta) is zero, that means you hit a local minimum for the error function, or a point where the error in the current state of the predictor is low, meaning it is (hopefully) a good predictor.

Note the f here is not the same as an activation function, as a neural network must perform a special kind of gradient descent called backpropagation. But overall the idea is similar.

So when you take partial derivative of the error expression above w.r.t. theta, where does it equal? You should note the activation functions derivative is involved which is how it’s used to measure error so to say.

Will typeset this properly when I get the chance.