I am working on a supervised machine learning problem where I have more than 10 probably 50 or 100 predicting label categories. Which type of model can be used to work on this type of problem in anaconda python.
Welcome to AI.SE @Par!
What you have might be either a multi-label or a multi-class classification problem. If the classes are disjoint (each example belongs to just 1 of the 50 classes), it's a multi-class problem. If not (so each example can belong to several classes at once), it's a multilabel problem.
Multi-label classification is usually handled by training a separate model for each of the labels. If you want to label a new point, you then ask each model what it thinks, and assign the union of the labels that the models suggest together. An alternative approach is to use something like a neural network, which can have many outputs. You can then have one output neuron for each possible label.
Multi-class classification can be addressed by using the multilabel techniques, but this is usually not a good idea. The three main approaches that are used are "one-v-one", "one-v-all", and "many-v-many".
- In 1-v-1, we train one model to discriminate each pair of classes (n(n+1)/2 models in total for n classes). To classify a new point, we ask each model which class it belongs to and assign a "vote" to the class the model returns. The class with the most votes overall is selected as the label for the new point. In case of a tie, we can report several possible answers to the user.
- In 1-v-all, we train one model to discriminate each class from all the other classes (n models in total for n classes). To classify a new point, we ask each model whether it belongs to the model's primary class or not. Ideally, just one model claims the new point. If more than one does, we can report a tie, or use some notion of classifier confidence to select a winner.
- In many-v-many, we train k models each of which is tasked with discriminating some subset of the classes from all the rest. To classify a new point, we ask all of these models to label the point. We then pick the class that that is most consistent with the results of all the models. This can also be done by deliberately constructing the models to form an error correcting code.
So which approach should you use? Well, Rifkin & Klautau's 2004 JMLR paper argues convincingly that the answer is to use one-versus-all classification. This is also pretty easy to do in most packages. For example, in Python's ScikitLearn, you can do it with OneVsRest. If you're not sure what to try, that's probably a safe bet.