Consider a Bayesian classifier used in spam e-mail filtering. It converts an e-mail to a vector, most of the time using the bag-of-words method. Although it learns first before getting employed, it can be made to work as an online system, i.e. it can be used to filter and learn from examples even after deployment.
Now, on the other hand, now comes the perceptron. It calculates a mean vector of spam and not spam, and then classifies them into the appropriate categories. The model adjusts the mean vectors each time it makes mistakes.
Now, comes neural nets, they too are capable of taking a vector-like bag of words or image pixels of dogs and cats and classify them into yes or no.
So, while designing and implementing them into the system, how to determine which one of the methods (Bayesian classifier, perceptron or neural network) is the most appropriate for a given situation or task? One factor to consider is the time complexity (or speed), but what are other factors, and how to rank them?