Let's say we have $K$ classes. For $K$ classes, we will be training $K$ different neural networks.
No, you still train one network.
With binary classification tasks, where you have only two mutually exclusive categories, like "yes/no" or "true/false", you can get away with a single output node with a sigmoid activation. The output of the sigmoid is interpreted as indicating one category for values $> 0.5$ and the other for values $\leq 0.5$.
With multi-class classification, you have $K$ outputs (one for each category). The problem, in this case, is that if the network gets the class wrong, in general, you cannot decide in one step which one of the other $K - 1$ categories is the correct one. So, the output is actually passed through an extra softmax layer, which outputs probabilities for each class.
But do we train one neural network at a time for all features, or do we train all $K$ neural networks at a time for one feature?
You present all features for each training example to the network at the same time. So, for $N$ features you have $N$ input nodes, and you feed all of them into the neural network.