Let's assume, that I have a neural network with few numerical features and one binary categorical feature. The network in this case is used for regression. I wonder if such a neural network can properly adjust to two different states of the categorical feature or maybe training two separate networks, according to these two states can be a better idea in the sense of smaller achievable error(assuming I have enough data for each of the two states)? The new model will use simple 'if statement' on the begining of the regression process and use proper network accordingly.

  • $\begingroup$ Not exactly what you're asking, but training a neural network multiple times from different initial states on the same data, then taking the average of running it through all the resultant networks nets a roughly 4% increase in accuracy. So I would guess this would improve it as well, so long as you don't over train. (I got the 4% stat from one of the cs231n lectures, I just can't remember which one, it's one of lecture 2-10 I think) $\endgroup$ – Recessive Sep 10 '19 at 4:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.