The term was introduced to the machine learning and computer science community by Rina Dechter in Learning While Searching in Constraint-Satisfaction-Problems (1986) , where she writes
Discovering all minimal conflict-sets amounts to acquiring all the possible information out of a dead-end. Yet, such deep learning may require considerable amount of work.
When deep learning is used in conjunction with restricting the level of learning we get deep first-order learning (identifying minimal conflict sets of size 1) and deep second-order learning (i.e. identifying minimal conflict-sets of sizes 1 and 2).
Our experiments (implemented in LISP on a Symbolits LISP Machine) show that in most cases both performance measures improve as we move from shallow learning to deep learning and from first-order to second order.
However, note that the term deep learning was used before 1986 in other contexts, for example, in . Moreover, note that Rina Dechter did not use the term in the context of neural networks, which was probably used later.