While reading source code related to RNN encoders, I've come across the term mask as input to the encoder. What exactly is it?
Masks in Recurrent Neural Networks are used to transform variable-length inputs to one general length. Therefore we use padding and masking together.
Padding: Usually we create a vector for every sentence in the dataset initialized with 0s and the length of the longest sentence in the dataset. Then we fill the mask with 1s for every position the sentence has words in.
Masking: Now we need to inform the network that some values in the vector are actually padding and should not be used since they have no information.