After reading on Transposed Convolutions and Fully Convolutional Networks in the d2l book (14.10 and 14.11), I wondered about the visualization of transposed convolutions. As you probably know, randomly initialized convolutional kernels learn to detect different shapes, see e.g. Fig. 8.1.1 from p.271 of the book: p.271 in the book The obvious question I then had is: What do the trained/learned transposed convolutions look like?

Firstly, initialization seems to play a much larger role; in the book, they are initialized so that they do bilinear interpolation, see enter image description here But then, after training, if I plot them again, all of the transposed convolutional kernels still look the same (yet their parameters changed slightly, but the network seemed to like to bilinear interpolation a lot).

Furthermore, I tried training the model in 14.11 from the book with randomly initialized transposed convolutions and failed doing so.

What is known about transposed convolutions? Is this their "optimal" initialization? Was there some discovery that they learn different kernels just like usual convolutions do?

Thanks a lot for any hint!



You must log in to answer this question.