Im implementing a neural network framework from scratch in C++ as a learning exercise. There is one concept I don't see explained anywhere clearly: How do you go from your last convolutional/pooling/something layer which is "3 dimensional" to your first fully connected layer in the network?
Many sources say, that you should flatten the data. Does this mean that you should just simply create a $1D$ vector with a size of $N*M*D$ ($N*M$ is the last conv. layer's size, and $D$ is the number of activation maps in that layer) and put the numbers in it one by one in some arbitrary order?
If this is the case I understand how to propagate further down the line, but how does backprogation work here? Just put the values in reverse order into the activation maps?
I also read, that you can do this "flattening" as a tensor contraction. How does that work exactly?