The reason is that when using a Convolutional layer, you select the size of the filter kernels, which are independent of the image/layer input size (provided that images smaller than the kernels are padded appropriately).
When using a Dense layer, you specify the size of the layer itself and the resulting weight matrix is a function of both the size of the Dense layer and the upstream layer. This is because each neuron in the upstream layer makes a connection to each neuron in the Dense layer. So if you have 50 neurons in the upstream layer and 20 neurons in the Dense layer, then the weight matrix has 50x20=1000 values. Those weights are what get determined during the training phase and so those layer sizes are fixed.
Now, the output of a CNN layer is a number of images/tensors (specified by the number of filters chosen) whose size is determined by the kernel size and any padding option chosen. If those feed into a Dense layer then that fixes the the size that those images can be (because of the reason given in the previous paragraph). On the other hand, if no Dense layer is used in the whole network then the input to the first CNN layer can be any size because the weights are just the individual parameters of the filter kernels, and the filter kernels remain the same size regardless of the input tensor size.