You are quite correct. If you have properly followed the Cross Validation procedure and selected the best model indeed, then you can use the CV set as the training set for the final model. And no it will not cause your hypothesis to worsen (for that set maybe, but not for new examples) if you have selected the model correctly. In-fact you may use the entire 100% of the data-set.

Justin Johnson a TA at Stanford University answered a similar type of question on training CNN's using 100% of the data-set. He said that if you had enough computational resources and want to squeeze that extra 1% or 2% accuracy from your model you can use the entire data-set after model selection.

You are quite correct. If you have properly followed the Cross Validation procedure and selected the best model indeed, then you can use the CV set as the training set for the final model. And no it will not cause your hypothesis to worsen (for that set maybe, but not for new examples) if you have selected the model correctly. In-fact you may use the entire 100% of the data-set.

Justin Johnson a TA at Stanford University answered a similar type of question on training CNN's using 100% of the data-set. He said that if you had enough computational resources and want to squeeze that extra 1% or 2% accuracy from your model you can use the entire data-set after model selection.

NOTE: As @NeilSlater pointed out, if you need the model for reporting purposes you should only use 80% of the data-set, otherwise you'll lose your only source for unbiased model verification. But if you are looking to deploy the model on field you can use 100% of the data-set.