# Why does CNN forward pass take longer compared to MLP forward pass? [closed]

Let's take a 32 x 32 x 3 NumPy array and convolve with 10 filters of size 2 x 2 x 3 with stride 2 to produce feature maps of volume 16 x 16 x 10. The total number of operations - 16 * 16 * 10 * 2 * 2 * 2 * 3 = 61440 operations. Now, let's take an input array of length 3072 (flattening the 32 * 32 * 3 array) and dot it with a weight matrix of size 500 x 3072. The total number of operations - 500 * 3072 * 2 = 3072000 operations. The convolution takes 4-5 times longer than np.dot(w, x) even though number of operations is less.

Here's my code for the convolution operation:

for i in range(16):
for j in range(16):
for k in range(10):
v[i, j, k] = np.sum(x[2 * i:2 * i + 2, 2 * j:2 * j + 2] * kernels[k])


Is np.dot(w, x) optimized or something? Or are my calculations wrong? Sorry if this is a silly question...

• Programming questions are generally off-topic here. Please, ask this type of question on Stack Overflow. Here we focus on theoretical, philosophical and social aspects of AI. Please, read also our on-topic page ai.stackexchange.com/help/on-topic to know more about it.
– nbro
Sep 6 '20 at 20:43

In your case, np.dot() is a matrix-vector multiplication which internally calls such a highly optimized BLAS routine. Those libraries are implemented in low-level languages like Fortan or C, they exist for many years now and are still unbeaten in terms of speed.
You could try to replace your manual convolution by the SciPy's equivalent scipy.signal.convolve2d(), to make advantage of optimized libraries as well and get a speed up.