Is my flowchart a good representation of the perceptron learning algorithm?

Question

I made a flowchart for a simplified perceptron leaning algorithm.

Here is the process of the learning algorithm.

1. Initialize the weights first.

2. Get a training example randomly and make a prediction. If the prediction matches the ground-truth value, then get another training example. If the prediction doesn't match the ground-truth value, update the weights.

3. repeat step 2 until all predictions match the ground-truth value (or other stop criteria)

Is my flowchart a good representation? If not, what are the errors, and what might be improved?

Answer 1

It seems loosely reasonable but there are various things which are potentially unclear.

What exactly is a prediction, and is it deterministic or stochastic? First, if you are predicting a continuous value, you can never be "correct" - there will always be at least some very small deviation. This makes me assume that you are talking about making some discrete prediction, e.g. over some classes. In this case you would typically output a probability distribution over the different classes. If this is the case, again it's unclear what "correct" means. This makes me believe that the only way to interpret "correct" is that for any example, you deterministically output a single class, e.g. by taking the class with maximum probability, and then the prediction is considered correct when you output the correct class.

I think the biggest issue is with "all predictions correct". How do you check if all predictions are correct? Would you compute the predictions for all examples each iteration? Because that seems like the only possible way to check whether or not all predictions would be correct. More generally it's often not possible to have all predictions be correct (i.e. for an over determined problem).

Answer 2

It is possible that the neural network may never "correctly" predict the labels of your samples. There are obviously practical issues like an ineffective loss function or optimizers leading to instability or overfitting. But even in theory, given a finite-size network, you may not have the representational power to correctly label your data (assuming the weights are perfectly optimal).

Therefore, the problem is essentially your flow chart may never stop, so you need to maybe add a stopping criteria like the maximum number of epochs/iterations.

Answer 3

The flowchart you've shown is an faithful representation of the process your described in the written version at the end of your question, with one or two elements not included. Some other comments/answers have rightly raised points about possible instabilities when measuring prediction "correctness". However, if your method is already set, my main suggestions for improving the visualisation for new readers would be:

• Indicate how you're measuring a correct prediction Are you using a direct equality test (==) or is there a loss function being used that your interpreting with a threshold or some other heuristic?

• Include your other Stop criteria You could add another box between Initialise Weights and Get a training example that reads Check Stopping Criteria (or similar) to indicate that you're checking some other things to see if you should continue training. This could then link to the "Stop" node with a "Conditions Met" arrow or similar. You could then link the N output from Are all predictions correct? to this new node to indicate it's checked every training step.

• Aesthetic/Preference Changes:

  • Put the plot on one line As every box in this flowchart is designed to be visited at least once I'd put them all in one horizontal/vertical line and let the arrows do the guiding. This could give the reader less to do if they're trying to understand the overall flow of the project.

  • Shade the nodes Start/Stop "terminal" nodes have a different job than "process" nodes such as Update weights. Colouring or shading them differently might help the user follow the flow of the chart at a glance, and a colour-blind friendly palette can help make it useful to as many people as possible.