In a time series regression problem I'm predicting "change" rather than the actual intended value i.e Instead of:
time, feature 1, feature 2, ....
2020, 1, 3.3
2021, 1.5, 5.2
2022, 1.3, 6.1
now, y_pred_1, y_pred_2, ....
I'm feeding the model and predicting percent changes:
2020 0, 0
2021 0.5, 0.57 # 5.2 / 3.3 - 1 = 0.57 percent change
2022, -0.13, 0.17
Now a problem that I'm facing is that when I introduce a new feature of a much larger magnitude and variance my model diverges during training:
# new feature
2020, 3000
2021 4
2022 600
# Which is transformed to percent changes
2020, 0
2021, 0.99
2022, 149 # oof
I tried mean normalizing that particular feature before transforming it to the percent change domain but it still diverges. Any ideas what else I can try to introduce this new feature to the inputs? What are some best practices for such contrasted inputs?
To get a general idea of the model if it has something to do with this:
The model consists of 5 parallel flows that get added together at the end. Each flow has two pairs of CNNs with different kernel sizes followed by 1-3 LSTMs. The output shape of each flow is the same so they can be added together. Then after adding it's followed by two fully connected layers predicting the change for the given features.