3
$\begingroup$

There is an example related to perceptron learning, but I couldn't get it, I don't exactly know how to solve it.

There is a snippet from lecture notes.

What is the transformation between epochs?

$\endgroup$
  • $\begingroup$ I'm not understanding your question. Do you think that you could edit it to make it a bit easier to understand? $\endgroup$ – Mithical Dec 14 '16 at 8:22
  • $\begingroup$ @Mithrandir : I am also a bit confused, because don't know, how can I transform a tables(epochs) to anothers. $\endgroup$ – appkovacs Dec 14 '16 at 8:59
  • $\begingroup$ One pass through all the weights for the whole training set is called one epoch of training. $\endgroup$ – appkovacs Dec 14 '16 at 20:36
2
$\begingroup$

There is no transformation between epochs. One full iteration over the training set is considered an epoch.

Lets assume:

  • We're considering a gradient descent in a space without local optima. This means that if you'd plot the errors we calculate below, this plot has but one lowest point.
  • The perceptron is intended to model a linearely seperable function. So, we could viualize our data points as a scatter plot and can draw a clear line through the scatter plot. Everything above/below the line should result in a true (1) output, everything on the other side should result in false (0).
  • We've set a learning rate (also called 'gain' or 'proportional change') upfront. This is a number between 0 and 1.

You train your perceptron on a data set. During one iteration over this entire training set, you:

  • Calculate an output and an error (delta between output and desired output) for a data point.
  • Adjust the input weights according to this error and the pre-set learning rate.
  • Apply the newly calculated weight.
  • Proceed to the next data point.

Transformation of weights happens during epochs, not between epochs.

Ergo, weights may be changed for each data point in the training set. Oftentimes you would need to iterate over the entire set several times, before the weights converge (stabilize / stop changing).

What use is the number of epochs?

The number of iterations is significant as a measure of efficiency of the current algorithm under the current learning rate, for the current dataset. In short: You could use it for comparisons between learning rates, training sets or algorithms.

$\endgroup$
  • $\begingroup$ Thanks for you answer! Could you please show an example with these calculations? $\endgroup$ – appkovacs Dec 16 '16 at 8:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.