3
$\begingroup$

Typical Feed Forward Neural Networks require a fixed sized input and output. So when you have variable sized input, it seems to be common practice to pad the input with zero vectors.

Why does it not seem to be common practice to have a "is_padding" attribute? That way the network can easily distinguish between padding and actual data? Especially considering input is commonly centered around 0 by subtracting the mean and using unit variance.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I would guess that maybe it's simply not necessary in common cases? If groups of inputs are often related to each other, such that some features being non-zero automatically implies a bunch of other features being "real" (even if they equal $0$), then a NN should be able to automatically learn that the $0$s are only padded if they are in a bunch together, all $0$? Really just a guess though. Intuitively, I do think your idea could be beneficial in some cases. $\endgroup$
    – Dennis Soemers ♦
    Commented Feb 4, 2019 at 16:58

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .