# How much can the addition of new features improve the performance?

How much can the addition of new features improve the performance of the model during the optimization process?

Let's say I have a total of 10 features. Suppose I start the optimisation process using only 3 features. Can the addition of the 7 remaining ones improve the performance of the model (which, you can assume, might already be quite high).

It depends on the used network as well as the feeding mechanism but let's give an example;

When working with LSTM, giving the time data (as an integer sequence) in addition to the time-series data(coming from features) dramatically increases the performance of the network.

[$$X_{0}$$,$$X_{1}$$, ...] $$\rightarrow$$ [[$$X_{0}$$,$$t_{0}$$],$$[X_{1}$$,$$t_{1}$$], ...]

If you go and look for the kaggle competition winner's notebooks, they do also create additional features based on the featured data.

Let's assume that the performance is already quite high on the three features so that you can predict those three features with high reliability. It would only make sense to increase the number of features if you would like to predict additional features!

Grouping the features of the concept class being learned can be used used to reduce the time required to learn the concept because it lowers the computation time of each sample and epoch. That's done almost always when recognizing complex movements or with other concept classes where the number of dimensions in the model is high. It is not a universal solution and care must be taken to configure the approach correctly.

If there is a local minimum that is directed along a particular axis in the loss surface which is not the global minimum and that axis happens to be the axis corresponding to the first feature the practitioner unwittingly chooses to optimize. The result will be that the other nine are learned immediately, yet the result is not optimal. That's an unlikely scenario but only one of many scenarios where the technique can trouble the learning objective.

Also, the interrelation between features of the concept class can be complex such that learning some of the features may cause the unlearning of others. Since a larger loop must be placed around the entire process so that the accuracy of the features learned earlier is further tested until all the factors affecting loss are adjusted to minimize it, the process can take longer in some cases.

The optimization of 10 features, as opposed to optimization of 3 features, will converge more slowly.

Let's say I have a total of 10 features. Suppose I start the optimisation process using only 3 features. Can the addition of the 7 remaining ones improve the performance of the model (which, you can assume, might already be quite high).