In a time series regression problem I'm predicting "change" rather than the actual intended value i.e Instead of:
time, feature 1, feature 2, .... 2020, 1, 3.3 2021, 1.5, 5.2 2022, 1.3, 6.1 now, y_pred_1, y_pred_2, ....
I'm feeding the model and predicting percent changes:
2020 0, 0 2021 0.5, 0.57 # 5.2 / 3.3 - 1 = 0.57 percent change 2022, -0.13, 0.17
Now a problem that I'm facing is that when I introduce a new feature of a much larger magnitude and variance my model diverges during training:
# new feature 2020, 3000 2021 4 2022 600 # Which is transformed to percent changes 2020, 0 2021, 0.99 2022, 149 # oof
I tried mean normalizing that particular feature before transforming it to the percent change domain but it still diverges. Any ideas what else I can try to introduce this new feature to the inputs? What are some best practices for such contrasted inputs?
To get a general idea of the model if it has something to do with this:
The model consists of 5 parallel flows that get added together at the end. Each flow has two pairs of CNNs with different kernel sizes followed by 1-3 LSTMs. The output shape of each flow is the same so they can be added together. Then after adding it's followed by two fully connected layers predicting the change for the given features.