Imagine I wish to classify images of digits from 0-9. Let's say I have trained the network to recognise '1'. If I were to train the same network to recognise '2', wouldn't the backpropagation process mess up the weights and biases for '1'?
Or do programs like Tensorflow allocate a new layer of neural network for different object classification? Thanks.
The answer is yes, retraining a neural network on a new dataset will alter its internal state such that it would no longer give the same output as before (in your case 'messing it up').
There are techniques to allow you to reuse sections of trained networks for other problems (this is called 'Transfer Learning'). Transfer Learning involves freezing the weights/biases for parts of the already trained network while adding extra new layers to the end of it and training those (with Back-propagation) for your new problem. For your digit recognition problem you would normally just train the network on all the digits at the same time.
Tensorflow is lower level than the idea of neural networks and layers. It is a library that allows you to write mathematical pipelines that implement the tensor calculations required to build a neural network. To do the sort of thing that you mentioned you would use a higher level library such as Keras (https://www.tensorflow.org/guide/keras) which now comes conveniently bundled with Tensorflow.
