The idea of dropout is that, at training time, with a certain probability $p_i \in [0, 1]$, the unit (or neuron) $i$ is dropped, $\forall i$, that is, the output of unit $i$ is set to zero so that $i$ does not affect the other units it is connected to, both during the forward and backward (or back-propagation) passes (or steps). At every mini-batch, you randomly drop usually different units, so, across different mini-batches (and consequently epochs), you do not always or necessarily drop the same units.
The title of the paper Improving neural networks by preventing co-adaptation of feature detectors emphasizes that dropout prevents the co-adaptation of the units (the feature detectors), so units attempt to detect certain features independently of other units, which reduces overfitting, that is, it improves the generalization ability of the neural network.
At test time, no unit is usually dropped. However, there is an approximation of a deep Gaussian process and Bayesian neural network that is based on the application of dropout at training and test times. This is called Monte Carlo dropout or, in short, MC dropout, for reasons you can understand if you read the paper Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning.
There's also the possibility to drop the connections between the neurons, which is called DropConnect, rather than the neurons themselves. These two approaches are slightly different, even though DropConnect can be seen as a generalization of dropout. In DropConnect, you do not switch off completely the units, but only the contributions of certain units to the output of other units. In dropout, you completely switch off certain units.
If you decided to deterministically (and manually) reduce the number of units before training, essentially, you would train a fixed smaller network, but this will not necessarily reduce overfitting or, more precisely, co-adaptation of the units. In dropout, you randomly select the units to drop, so, at every mini-batch (or epoch, depending on the implementation of dropout), you effectively train a random subset of the units of the original neural network and, because of this, it can be thought of as an ensemble of smaller neural networks.
The two papers Improving neural networks by preventing co-adaptation of feature detectors (2012) and Dropout: A Simple Way to Prevent Neural Networks from Overfitting (2014) have exactly the same authors, but the latter paper was published in the Journal of Machine Learning Research, while the former wasn't apparently published in any journal. In fact, Dropout: A Simple Way to Prevent Neural Networks from Overfitting does not even cite Improving neural networks by preventing co-adaptation of feature detectors, but it cites the master's thesis Improving Neural Networks with Dropout (2013) by Nitish Srivastava, who is one of the authors of dropout.