Which online machine learning technique to use for multi-class classification problem with multiple inputs?

Question

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. Another constraint is that the solution should allow for online learning with singular values or mini-batches, as the dataset is too big to load all the training data into memory.

I have tried the following method, but the predictions are not good.

I created an MLP or feedforward network with $4$ inputs and $127$ outputs. The inputs are being fed without normalization. The number of hidden layers is $4$ with $[8,16,32,64]$ units in each (respectively). So, essentially, this treats the problem like a sequence classification problem. For training, we feed the non-normalized input along with a one-hot encoded vector for that specific value as output. The inference is done the same way. Finding the hottest output and returning that as the next number in the sequence.

Answer 1

I suggest using Data Stream algorithms to try on your problems since you are asking for "online learning with singular values or minibatches as the dataset is too big too load all the training data into memory."

MOA is a good choice for these algorithms. Hoeffding Trees is also a good first choice to try.

Answer 2

If you prefer to use Python (rather than Java, which is used to implement MOA, which is suggested in the other answer), you might want to have a look at the Python's creme library, whose API is described at https://creme-ml.github.io/api.html, which is a library for incremental and online learning. In particular, you might be interested in the class OneVsRestClassifier.