1
$\begingroup$

I have a list of increasing numbers with no duplicates for example : 3,6,11 and so on. Difference or deltas between these numbers such as in above case : 3, 3, 5 are frequent and with greatest difference being 9. Can a neural network be used to predict next 3 deltas for next 3 numbers. Which type of neural network will be useful and what kind of limitations will there be ?
Note : List comprises of millions of numbers. These numbers and deltas are not random.

$\endgroup$

1 Answer 1

3
$\begingroup$

Yes, predicting deltas is often easier than predicting values directly. The most obvious way to perform this task would be to use a simple MLP with MSE loss. If the deltas are discrete, and there are only 9 possible values, a multi-label cross entropy loss would probably perform better than using MSE.

Depending on what the relationships between the numbers are, you might want to feed the last W numbers to the neural network, so that it can learn to use context. To do that, just make your input dimension W, and have your output dimension be 1. If you need a very long context window for good prediction, you should consider using an RNN.

One thing that would help to improve training is that the scale of the inputs shouldn't be too different (e.g. some inputs being ~10^5 and some being ~10^0). If you can rescale your inputs to always be close to 0, that's helpful. For example, if the rule which relates numbers to one another is +3, +5, +3, +5, ... you could always normalize the first input to be 0, and un-normalize after you get your neural network output, without any loss of information.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for quick response and answer. $\endgroup$
    – Akash
    Sep 28, 2022 at 3:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .