Isn't attention mask for BERT model useless?

Question

I have just dived into deep learning for NLP, and now I'm learning how the BERT model works. What I found odd is why the BERT model needs to have an attention mask. As clearly shown in this tutorial https://huggingface.co/transformers/glossary.html:

from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")

sequence_a = "This is a short sequence."
sequence_b = "This is a rather long sequence. It is at least longer than the sequence A."

encoded_sequence_a = tokenizer(sequence_a)["input_ids"]
encoded_sequence_b = tokenizer(sequence_b)["input_ids"]

padded_sequences = tokenizer([sequence_a, sequence_b], padding=True)

Output of padded sequences input ids:

padded_sequences["input_ids"]

[[101, 1188, 1110, 170, 1603, 4954, 119, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 1188, 1110, 170, 1897, 1263, 4954, 119, 1135, 1110, 1120, 1655, 2039, 1190, 1103, 4954, 138, 119, 102]]

Output of padded sequence attention mask:

padded_sequences["attention_mask"]
[[1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]

In the tutorial, it clearly states that an attention mask is needed to tell the model (BERT) which input ids need to be attended and which not (if an element in attention mask is 1 then the model will pay attention to that index, if it is 0 then model will not pay attention).

The thing I don't get is: why does BERT have an attention mask in the first place? Doesn't model need only input ids because you can clearly see that attention_mask has zeros on the same indices as the input_ids. Why does the model need to have an additional layer of difficulty added?

I know that BERT was created in google's "super duper laboratories", so I think the creators had something in their minds and had a strong reason for creating an attention mask as a part of the input.

Answer 1

This is just an implementation issue. One reason is the Huggingface implementation (which is not the original implementation by Google) wants to strictly separate the tokenization from the modeling. It is a convention that the input sequences are zero-padded, but in theory, it does not have to be so. In the Huggingface implementation, you use a different tokenizer that would pad the sequences with different numbers and still get valid masking.

You are right that you can infer the mask from the input IDs at the very beginning (if you know the pad ID), but you need to explicitly use the mask in every single layer. Each layer returns a 3D tensor of floats from which you cannot say what the padded positions are, you need to have the explicit mask when calling the next layer. I guess that having the mask everywhere makes the API more consistent.

Answer 2

It makes sense to me that BERT does not require attention mask. BERT is a bi-directional encoder. Each word in a sequence is allowed to "attend" other words from both left and right sides. Attention mask would only make sense if the encoder is uni-directional, or in case of a decoder, where each word is only allowed to attend the words before it.

I also think that this is related to Huggingface implementation. BERT is an encoder. The pytorch implementation of an encoder can be found here. As you can see in the "forward" function there, the argument "mask" (in this case it refers to attention mask) is only optional.

Answer 3

In huggingface implementation, they have made the Bert implementation generic. So that It could be used as both an encoder or an decoder.

Example, for a transformer to act as decoder, only two things that's required is :

1. masking future tokens.
2. Also do cross attention based on "supplied encoder representations".

Check the code for transformers.BertLMHeadModel where Bert is used as decoder while transformers.BertforMaskedLM uses it as our regular encoder.