# Convolutional Sequence to Sequence Learning kernel parameters

I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look at the relevant part of the paper below.

Let's say the input to the kernel X is k*d (say k=5 words of d=300 embedding dimenisonality). Therefore the input is 5*300. In a computer vision task a kernel would slide over parts of the image, in NLP you usually see kernel taking up the whole width of the input matrix. So I would expect kernel to be m*d (e.g. 3*300 - slide over 3 words and look at their whole embeddiings).

However, the kernel here is of dimensionality 2d x kd which in our hypothetical example would be 600*1500. I don't understand how this massive kernel would slide over an input that is by far lower dimensional (5*300). In computer vision you could zero-pad the input, but here zero-padding would basically turn the input matrix into mostly zeros with only a handful of meaningful numbers.

Thanks for shedding some light on it!

They are doing a matrix multiplication: consider $$y = Ax, y \in \mathbb{R}^m, x \in \mathbb{R}^n, A \in M_\mathbb{R}(m,n)$$. In the paper $$x$$ is a concatenation of $$k$$ elements of $$\mathbb{R}^d$$, so $$x$$ is long $$kd$$; $$y$$ is long $$2d$$.
First for clarity I want to differ "convolutional kernel" and "filter" here, let's say a filter has several convolutional kernels. Second in NLP 1D-convolution is mostly used, so different from CV, which typically use a $$k \times k$$ convolutional kernel, in NLP generally the kernel size is only $$k$$
Having listed above hope things could become a little clearer: If we adopt the same concepts as we seen in typical CV scenario, per my understanding, the size of convolutional kernel is only $$k$$, but the count of kernels, or the dimension of new "channel", is set to $$2d$$. One kernel maps $$k \times d$$ elements into one element, $$2d$$ kernels together convert the input to a vector of which the dimention is $$2d$$
As the calculation process is mapping a matrix $$\mathbb{R}^{k \times d}$$ to a vector $$\mathbb{R}^{2d \times 1}$$, if expressed in a form of matrix multiplication, or in a full-connected view, we can firstly expand the input matrix, concatenate all rows together to get a new big vector of which the dimension is $$\mathbb{R}^{kd \times 1}$$, then multiply a weight matrix $$\boldsymbol{W} \in \mathbb{R}^{2d \times kd}$$ could get the final vector
The key problem that roots the confusion is beause, in my opinion, the "kernel" imported here is a little different from the same concept used in a typical CNN. Again, if we describe in a traditional CNN terminology, the kernel size should be $$k$$, and there are $$2d$$ kernels applied.