# How is dropout applied to the embedding layer's output?

model = tf.keras.Sequential([
tf.keras.layers.Embedding(1000, 16, input_length=20),
tf.keras.layers.Dropout(0.2),                           # <- How does the dropout work?
tf.keras.layers.Conv1D(64, 5, activation='relu'),
tf.keras.layers.MaxPooling1D(pool_size=4),
tf.keras.layers.LSTM(64),
tf.keras.layers.Dense(1, activation='sigmoid')
])


I can understand when dropout is applied between Dense layers, which randomly drops and prevents the former layer neurons from updating parameters. I don't understand how dropout works after an embedding layer.

Let's say the output shape of the Embedding layer is (batch_size,20,16) or simply (20,16) if we ignore the batch size. How is dropout applied to the embedding layer's output?

Randomly dropout rows or columns?

After an Dense Layer, the Dropout inputs are directly the outputs of the Dense layer neurons, as you said. After your embedding layer, in your case, you should have rate * (16 * input_length) = 0.2 * 20 * 16 = 64 inputs set to 0 out of the 320 scalars inputs. These 64 dropped inputs are randomly selected in the 20x16 grid. Note that the Dropout rescales the non dropped inputs by multiplicating them by a factor $$\frac{1}{1-rate}$$.