1
$\begingroup$

That is, if some of the inputs to a neural network can be calculated by a pre-determined function whose variables are other inputs, then are those specific inputs useless?

For example, suppose there are three inputs, $x_1$, $x_2$ and $x_3$. If $x_3$ is determined by function $x_3=f(x_1,x_2)$, then will $x_3$ be useless for training a neural network?

$\endgroup$

2 Answers 2

2
$\begingroup$

No it is not useless.

The relationship may not be obvious, and having the data will allow the network to learn this 𝑓 relationship.

Further, even if 𝑓 is obvious, networks are so sample inefficient that more (non-noisy) data is always helpful. In fact, the common practice is to train for hundreds of epochs on the same exact samples - because we can not learn quickly enough from seeing them only once.

That said, there are some cases where data is harmful. For example, if we have an imbalanced dataset, adding more samples to exacerbate that imbalanced may be a bad idea.

But in general, this added data will still be of use.

$\endgroup$
1
  • $\begingroup$ What’s wrong with imbalance? $\endgroup$
    – Dave
    Commented Mar 3, 2023 at 11:52
0
$\begingroup$

There is a difference between adding more samples to the data (rows), and adding more features (columns). In this case we are talking about more features.

If the function $f$ is trivial, feeding extra columns doesn't hurt but doesn't bring any benefits either. And the training is a bit slower, since extra gradients needs to be calculated. If it is non-trivial, it could be the case that this helps the network train faster and maybe you can even use a smaller network.

It is common to pre-process data before feeding it to the network, for example transforming a time-feature into two season-features via sin and cos transformations.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .