0
$\begingroup$

Imagine that you want to train a deep learning model for forecasting/transforming data and have a training dataset with endpoints 0 and 1000. Now, Imagine receiving a new dataset with endpoints 0 and 2000. If I scale it as before, my new scaled data has endpoints 0 and 2, but I didn't train my model for those endpoints. In this case, how have I to behave? Do you have any suggested books about data preprocessing for deep learning models?

The model should operate online, so we don't know the endpoint of the following dataset (it be 2000 as it can be any number)

$\endgroup$

1 Answer 1

0
$\begingroup$

Without some other prior knowledge about the data, it's hard to expect your model to perform effectively. Your model models the training data -- you can't really expect your model to perform well on data that it's never seen before.

This might be ok if you're using a simpler model (e.g., a linear classifier), but for deep learning models you should make sure that the distribution of data during training matches the distribution during testing.

If you do have prior knowledge about the data, then you can mitigate this with clever preprocessing or data augmentation. For example, maybe you really only care about the relative differences between different input features. In that case, the ranges of the inputs don't matter as much, and you can e.g., just give the model the difference of the inputs or augment your input to include samples where you adjust all your inputs by a constant amount (that way, your model learns to handle different ranges, but the relative values are still the same).

A more concrete example of this is in time-series data: Augmenting time series data to be stationary. Source: https://otexts.com/fpp2/stationarity.html

Image source: https://otexts.com/fpp2/stationarity.html

Here, although the range of the data changes over time (the top two plots), you can augment the data such that the model input has similar ranges regardless of the range of time you're considering (the bottom two plots).

As you might guess, this is highly domain dependent. You should consider what kind of information your model really needs to make predictions

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .