Tomas Mikolov's mention of gradient clipping in a single paragraph of his PhD thesis in 2012 is the first appearance in the literature.
The first source (Mikolov, 2012) in the Deep Learning book is Mikolov's PhD thesis and can be found here. The end of section 3.2.2 is where gradient clipping is discussed, only it's called turncating.
... The exploding gradient problem has
been described in .
A simple solution to the exploding gradient problem is to truncate values of the gradients. In my experiments, I did limit maximum size of gradients of errors that get accumulated in the hidden neurons to be in a range < −15; 15 >. This greatly increases stability
of the training, and otherwise it would not be possible to train RNN LMs successfully on
large data sets.
 Y. Bengio, P. Simard, P. Frasconi. Learning Long-Term Dependencies with Gradient Descent is Difficult. IEEE Transactions on Neural Networks, 5, 157-166, 1994.
A search of the referenced paper  shows that it does describe the problem as Mikolov said, but it does not present gradient clipping as a solution.
So I had a look at the second source Deep Learning mentioned: On the difficulty of training Recurrent Neural Networks. It directly cites Mikolov as having proposed clipping:
We would make a final note about the approach proposed by Tomas Mikolov in his PhD thesis (Mikolov, 2012) (and implicitly used in the state of the art results on language modelling (Mikolov et al., 2011)). It involves clipping the gradient’s temporal components element-wise (clipping an entry when it exceeds in absolute value a fixed threshold). Clipping has been shown to do well in practice and it forms the backbone of our approach.
I thought about emailing Mikolov to verify that his thesis was the origin of the idea. But then I noticed that he is a co-author of this paper which cites him as proposing it! Though I still wonder if it was commonly used in practice before
even though it had not been published.