So I basically have a $n$ classes. I have an input. My data is organised in the following way: each input has a label, this label is 2 classes. It can be twice the same class, or two different classes.

So I want to output a vector with only 0s, 1s and 2s which sum is 2. Similarly to what we would do if the label is only one class in a regular classification model. I'm just a bit confused about how we would handle the standard the prediction is the argmax because here we would have to make arbitrary rules about when to predict 2 classes and when to predict two distinct classes (like in this case [0, 0.5, 0.4, 1.1, 0, 0], it's a bit hard to determine if class 4 is predicted twice or if it's one time class 2 and one time class 4, etc ...)

Is this a thing, is there a scientific term about it? Am I not supposed to do that because it breaks something fundamental about classifiers? I'm planning on training Neural Networks, XGBoost, Random Forest, Extremely Randomized Trees and comparing the different methods (are there other methods where this would be possible? basically predicting a vector whose sum is 2).

  • $\begingroup$ What is the value of $n$? If it is not too large, you can create all combinations of it, and use regular one-hot vector. For example, if you have 4 classes, consider using $2^4$ = 16 output nodes $\endgroup$ Mar 4, 2023 at 1:38
  • $\begingroup$ Very good idea although $n$ is very large in my case, around 30, making this solution probably too computationnally expensive. $\endgroup$ Mar 4, 2023 at 11:53
  • $\begingroup$ If computation allows, I suggest setting the baseline as the One-vs-All problem first. In your case, train $n$ models, each for a specific class. $\endgroup$ Mar 4, 2023 at 13:44
  • $\begingroup$ @Minh-LongLuu So training $n$ models that output a vector of size 3? How do I compile the outputs, it doesn't seem as straightforward as doing an argmax .. or maybe it is easy and i'm missing it? $\endgroup$ Mar 4, 2023 at 15:38
  • $\begingroup$ Or maybe $3n$ models, 3 because for each category i have three models (binary classification): class 0 for this or something else, class 1 for this or something else and class 2 for this or something else. But I would have the same problem of compiling the outputs .. $\endgroup$ Mar 4, 2023 at 15:42


You must log in to answer this question.

Browse other questions tagged .