0
$\begingroup$

What is the impact of the number of features on the prediction power of an ANN model (in general)? Does an increase in the number of features mean a more powerful prediction model (for approximation purpose)?

I'm asking these questions because I am wondering if there is any benefit in using two variables (rather than one) to predict one output.

If there is a scientific paper that answers my question, I would thank you.

Improve this question
$\endgroup$
2
  • $\begingroup$ I'm not sure 1. if you're looking for a theoretical result on the approximation capabilities of neural networks when the number of features changes, or 2. if you're just asking if adding features can, in practice, make your NN produce more accurate results (e.g. in terms of accuracy). Can you clarify this? $\endgroup$
    – nbro
    Dec 4, 2020 at 14:14
  • $\begingroup$ yes, I mean 1 and 2 $\endgroup$
    – Yazan Alatoom
    Dec 4, 2020 at 18:19

1 Answer 1

Reset to default
0
$\begingroup$

Based on the clarifications given in the comments on the original question, i will try my best to give an answer.

If the number of features increases, does the approximation capability of an ANN improve theoretically speaking? It depends on whether this additional feature adds 'usefulness' to the data that was already supported. If you add a feature that is already in the dataset, it obviously will not increase the approximation capabilities. If you add a feature that is not yet in the current set of features and does influence the thing you are trying to approximate, then yes! There are some exceptions in this case of course, such as what if features are multicorrelated etc. It is not super important or measurable in ANNs, but statistics has a bunch of theory on this stuff. Simple statistical multiple regression has 4 assumptions and also checks for multicollinearity to see whether it is possible get 'useful' results from the data etc. As ANNs is super general, this is not applicable, but you could argue that similar practices could be looked into when seeking the best possible 'theoretical' validity of the features that you are using.

Does adding more features make your ANNs produce more accurate results (in practice)? Again, it depends. If you indeed added a useful feature, then the possibility exists that your ANN will in practice be able to produce more accurate results. But if your ANN is not able to converge to a solution, then your results will be gibberish. It is more likely that a network is unable to converge if you just throw data at it. More data requires bigger networks etc. So it is highly dependent on your training method. AKA, if your added feature is useful and the method you use is sufficient, then I'd argue that, yes, adding a feature will most likely make your ANNs produce more accurate results.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ thank you for your detailed answer $\endgroup$
    – Yazan Alatoom
    Dec 5, 2020 at 14:28
  • $\begingroup$ Are you sure that the approximation capabilities of neural networks change depending on the number of features? According to which "theory" is this true? For example, does the VC dimension of a neural network change if you change the number or type of features? $\endgroup$
    – nbro
    Dec 5, 2020 at 15:48
  • $\begingroup$ I am not very knowledged about the VC dimensions theory or similar theories that apply more generally to NNs. If you are, please do elaborate on them. My approach here was to give a semi-intuitive understanding of when to expect 'more' approximation capabilities. I guess you could argue that my first stance on the 'theoretical' side could also be put as more 'practical', but I don't think it matters too much here. $\endgroup$
    – Robin van Hoorn
    Dec 6, 2020 at 9:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .