Suppose I have a neural network with 5 inputs: [A,B,C,D,E]

There is only 1 output. The expected accuracy of the model should increase when all 5 inputs are available, but often not all 5 inputs are available. For example, I might have case where I only have a variable number of the inputs, e.g. [A,B,C,-,-], [A,-,-,-,E], [-,B,-,D,-], [A,-,-,-,-], [-,-,C,-,-], [A,-,C,D,E], etc.

In such a situation, what is the best way to train or build the neural network? Are there any specific approaches or architectures recommended for this type of problem?

A couple ideas that come to mind include:

  1. Double the number of inputs to the neural network by including a second "binary input vector" that determines whether the input variable is present or not. For example, the binary input vector for the inputs [A,-,C,-,E] would simply correspond to [1,0,1,0,1], which could be fed into the neural network as well. The outstanding question is how does one treat the undefined variables with "-" as placeholders in such an example...perhaps defaulting to 0 for "-" is one naive but simple way when coupling the binary vector.

  2. Build and train separate neural network for every combination of [A,B,C,D,E] — this could certainly be implemented, but would be a brute force approach that requires a lot of training and be rather inefficient. For 5 variables, this would require 31 separate neural networks, and would scale poorly as the number of potential input variables increase

Any and all insights into this problem are appreciated!

  • $\begingroup$ Some domain knowledge would help here, what is the nature of these inputs? For example are they real numbers, with Gaussian distribution? Or are they always non-negative? $\endgroup$
    – NikoNyrh
    Commented Mar 15 at 13:00
  • $\begingroup$ Neural network or not the typical way of dealing with missing variables is imputation (mean, mode, etc). Increasing the dimensionality of the problem (by adding variables) is tempting the curse of dimensionality. $\endgroup$
    – foreverska
    Commented Mar 15 at 13:46
  • $\begingroup$ Oh, and is each input a single number, or a tensor? And do they have any co-correlation or are they basically independent from each other? $\endgroup$
    – NikoNyrh
    Commented Mar 15 at 16:45
  • $\begingroup$ @NikoNyrh each variable input (e.g. A, B, C, D, E) corresponds to a real number between -1 and +1. The exact distribution of each variable is not precisely known and likely not Gaussian (@foreverska thus perhaps it's best to not assume anything about its distribution) $\endgroup$
    – user18959
    Commented Mar 16 at 0:22
  • $\begingroup$ @foreverska is there any further context that would be helpful? For added reference, these inputs (i.e. A, B, C, D, E) correspond to specific types of measurements. For example, A = height, B = age, etc. and thus I don't believe randomly ordering inputs would be a successful approach $\endgroup$
    – user18959
    Commented Mar 19 at 3:23


You must log in to answer this question.