What is a "generalized" machine learning algorithm?

Question

What does it mean when it is said that Machine Learning algorithm results can be "generalized"?

I don't understand what "generalized" algorithms, routines or functions are.

I have searched dictionaries and glossaries, and cannot find an explanation. Also, if anyone can tell me where a good source for this type of thing is? I am writing about AI and ML.

Answer 1

Actually you have used 2 terminologies there:

• The first one is that Machine Learning algorithm results can be "generalised". This refers to how well your trained Machine Learning model will perform on previously unseen data (test set or implemented on field). This is particularly not easy as data trends may change over time resulting in loss of accuracy. There are various methods to implement this like (having a cross validation set and a test set, which comes under the broad scheme of k-fold cross validation)
• The second you mentioned is '"generalised" algorithms, routines or functions'. Most Machine Learning algorithms can be applied to a broad range of problems. For example the training of a NN is generally done by backprop which is universally applied to all NN's. Similarly, you can use CNN to find features of local interest (i.e. local dependencies) in anything that can be represented in a pictorial form (strings of DNA). Also combinations of CNN and RNN are being used to solve many problems. Thus, only a basic generalised algorithm is being applied to a lot of problems. NOTE: I have never seen any one use it in this context, but practically it happens.

Here are a few resources for general reading purposes (not mathematical):

• Overfitting and Underfitting With Machine Learning Algorithms
• Generalization and Overfitting

Answer 2

The brief answer is: generalized machine algorithm is an algorithm that can do well and give good results in new data that never seen before

Answer 3

Using the adjective generalized with algorithms, routines, or functions is obscure. The more appropriate term is generic.

Generics began with early loaders, before full fledged operating systems were developed. The idea was that a module could be loaded into computer memory and the loader was able to perform its function whether the information being loaded was a program, a routine called from a program, an update of the loader, or data to be used by a program. This abstraction was mostly the brainchild of mathematicians such as Kurt Gödel and John von Neumann, both of whom contributed to the same kind of generic representations in the field of abstract algebra.

The ideas of generics spread to the development of file systems. When LISP was developed, the deliberate blurring of the distinction between programs and data was built into the low level functions for working with both. Functional programming developed into higher levels of abstraction in the LISP community.

The idea of generics was introduced into C++ by James Coplien using the term idiom to solve maintainability and readability issues with early clumsy attempt at generic programming using the C pre-prossessing directives. The first highly usable form of generics appeared in Alexander Alexandrovich Stepanov's standard templates library (STL), developed at SGI. Frameworks like COBRA and DCOM were designed to permit generic messaging.

Generic programming constructs spread to Java and then to other languages. Many algorithms are written generically, such that the actual object types with which the algorithm works are not known until the template is employed at compile time.

The multilayer perceptron is a generic learning component, parameterized by a specific numerical representation for forward signal propagation, numerical representation for backward corrective signalling, layer depth, widths and activation functions for each layer, learning rate, and other hyper-parameters. (What are called network parameters are not parameters of the generic network but parameters of that control the mix of inputs into each layer and become the primary data output of the learning process.)

The other artificial network types are similar in this respect.

Even the cell types can be a type parameter in a network. See Neural Network Cell (Node) Types.

The question, "What does it mean when it is said that Machine Learning algorithm results can be generalized?" cannot be intelligently answered. Generalizing results is too general a description of what can be done to results to mean anything. It may just be rhetorical.

There are no universally, "good source[s] for this type of thing." There are many excellent sources for very specific things buried within a hundred times the volume of marginally correct and educationally barren sources. If you don't have the time to scrutinize what you are reading to carefully extract the gems, then it might be wise to attend some courses at a well respected university to get a more solid foundation from a more reliable source than the Internet.

Answer 4

A machine learning model is said to "generalise" when it performs equally well on both train and test datasets.

For any supervised machine learning algorithm to work well, you train it on a large dataset (train) and evaluate its performance on a dataset whose probability distribution is similar to that of train set but not a part of the train set. This set is called "test" set , then the performance is evaluated on this set, if both train and test accuracies are almost same then the model is said to generalise well, if the training accuracy is much higher than that of test accuracy then the model in some way "memorises" the train set and called "overfitting".