# How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer.

If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a network. How is the varying input size reconciled with the fixed size of the input layer of the network? In other words, how is such a network made flexible enough to deal with an input that might be anywhere from one word to multiple pages of text?

If my assumption of a fixed number of input neurons is wrong and new input neurons are added to/removed from the network to match the input size I don't see how these can ever be trained.

I give the example of NLP, but lots of problems have an inherently unpredictable input size. I'm interested in the general approach for dealing with this.

For images, it's clear you can up/downsample to a fixed size, but, for text, this seems to be an impossible approach since adding/removing text changes the meaning of the original input.

• can you clarify what you mean by downsampling to a fixed size? How is downsampling done? Commented Jul 28, 2019 at 0:51
• Commented Jul 28, 2019 at 1:02
• Whereas the original question is fairly open-ended, the answers focus primarily on NLP. However, I stumbled on this question while looking how to do variable size image inputs for a CNN. Variable size inputs are indeed possible for a convolutional approach - albeit with some caveats, and the stats.stackexchange link above explores that related, alternate line of inquiry. Commented Jan 17, 2020 at 6:25
• A similar question: How to input Scikit learn MLP classifier with variable length of input data. Commented Dec 9, 2021 at 7:44
• I'm curious why no one mentions Fully Convolutional Networks and Spatial Pyramid Pooling. Commented Aug 1, 2022 at 19:29

Three possibilities come to mind.

The easiest is the zero-padding. Basically, you take a rather big input size and just add zeroes if your concrete input is too small. Of course, this is pretty limited and certainly not useful if your input ranges from a few words to full texts.

Recurrent NNs (RNN) are a very natural NN to choose if you have texts of varying size as input. You input words as word vectors (or embeddings) just one after another and the internal state of the RNN is supposed to encode the meaning of the full string of words. This is one of the earlier papers.

Another possibility is using recursive NNs. This is basically a form of preprocessing in which a text is recursively reduced to a smaller number of word vectors until only one is left - your input, which is supposed to encode the whole text. This makes a lot of sense from a linguistic point of view if your input consists of sentences (which can vary a lot in size), because sentences are structured recursively. For example, the word vector for "the man", should be similar to the word vector for "the man who mistook his wife for a hat", because noun phrases act like nouns, etc. Often, you can use linguistic information to guide your recursion on the sentence. If you want to go way beyond the Wikipedia article, this is probably a good start.

• What is the difference between what you called "recursive NNs", and RNNs? R is for recursive... Also, a few citations and pointers would be useful. Commented Sep 24, 2016 at 2:13
• R is for recurrent. Recurrent means adding input linearly to the same NN again and again. Recursive means inputting the data along a tree structure. Commented Sep 25, 2016 at 9:29
• @BlindKungFuMaster 0 padding with mask for MLPs is it good and does not affect badly the accuracy ? Commented Feb 25, 2019 at 8:13

• RNN
• recursive NN

so I will add another possibility: using convolutions different number of times depending on the size of input. Here is an excellent book which backs up this approach:

Consider a collection of images, where each image has a different width and height. It is unclear how to model such inputs with a weight matrix of fixed size. Convolution is straightforward to apply; the kernel is simply applied a different number of times depending on the size of the input, and the output of the convolution operation scales accordingly.

Taken from page 354, 9.7 Data Types, 3rd paragraph. You can read it further to see some other approaches.

• It occurs to me that this approach will only work if 1) the aspect ratio (AR) of all the input images is the same, 2) you rescale all images to a given AR, or 3) you zero-pad images to force a given AR. Commented Nov 1, 2018 at 11:34
• @Salvador Dali can "0" padding with mask be a good solution for MLPs? Commented Feb 25, 2019 at 8:15
• I can't find the text you cite in your answer in the book. Maybe it was present originally and has been removed later? Page 354 currently has a paragraph which starts similarly, but never uses variable number of layers (but variable output size instead). Commented Apr 21, 2019 at 20:39
• FWIW, many (most?) implementations of modern CNNs for vision domains end with an adaptive pooling layer (e.g., adaptive averaging). So even though they apply convolutions a fixed number of times, and although they usually have a minimum input size, they usually don't have any maximum input size, nor do they usually have any further restrictions on aspect ratio, etc. I can imagine doing something similar for 1D varied-length data. I haven't read any papers on adaptive pooling; I just know that it's done a lot in practice. Commented Sep 13, 2022 at 21:03

In NLP you have an inherent ordering of the inputs so RNNs are a natural choice.

For variable sized inputs where there is no particular ordering among the inputs, one can design networks which:

1. use a repetition of the same subnetwork for each of the groups of inputs (i.e. with shared weights). This repeated subnetwork learns a representation of the (groups of) inputs.
2. use an operation on the representation of the inputs which has the same symmetry as the inputs. For order invariant data, averaging the representations from the input networks is a possible choice.
3. use an output network to minimize the loss function at the output based on the combination of the representations of the input.

The structure looks as follows:

Similar networks have been used to learn the relations between objects (arxiv:1702.05068).

A simple example of how to learning the sample variance of a variable sized set of values is given here (disclaimer: I'm the author of the linked article).

one other big thing which gives this ability of dealing varying input sizes, which is the most important reason that they do such, is their independence of model of sequence input and out sizes. so there should be no layer which one of its input or out sizes is input or out sizes of your sequence, defined. like in rnn family of neural networks, which are used for varying size sequences, as they rely on dimension of each sequence item. also we can see it in transformers which are also used for varying size sequences we can this independence, as its been demonstrated in this repo of multivariate transformers for forecasting(note in this models usually its only dependent on model max input size which is needed in some embeddings, but note the network itself is independent of sequence input and output sizes).