How to design a neural network when the number of inputs is variable?

Question

I'm looking to design a neural network that can predict which runner wins in a sports game, where the number of runners varies between 2-10. In each case, specific data about the individual runners (for example, the weight, height, average speed in previous races, nationality, etc) would be fed into the neural network.

What design would be most advantageous for such a neural network?

Essentially this is a ranking problem where the number of inputs and outputs are variable.

Answer 1

The best option in your case would probably be zero-padding or padding up. This is simply zeroing out inputs for cases in which there is no data. It's done a lot on the borders of images for CNNs.

Alternatively, you could just use an RNN, which can handle your variable-length inputs with ease.

Answer 2

I think, the proposed in the other answer CNN and RNN is a bad choice for this particular problem.

The input is the unordered sequence of the features, corresponding to each runner, so the input is essentially a set structure without the notion of order and locality. If the runners are assigned a number in random order, there is no sense as runner 1 is close to 2, or runner 3 precedes runner 4.

Therefore, CNN or RNN seem to be a bad choice since the add an inductive bias irrelevant to the data.

The question is pretty old, and at that time Transformer architecture was just invented, but it is the case, where it can be particularly well applied. Some points to note:

• Since there is no order, there is no need for positional embeddings
• Sequences are short, hence $O(N^2)$ complexity with respect to the length of sequence is small
• There is no need for decoder

Overall, I would suggest to stack several encoder blocks and predict ranks from the resulting embeddings.

As a loss, choose some reasonable choice from mentioned in the paper Ranking Measures and Loss Functions in Learning to Rank.