I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient calculations and relationships between gradients.
I have a problem in understanding what exactly symbol-to-number differentiation is. Does it map directly every variable in the backprop to its relevant gradient? If yes, how does it do this without knowing about the rest of the computational graph?
If the question is unclear, to increase context - TensorFlow uses symbol to symbol differentiation whereas torch uses symbol to number (apparently).
Came across this in section 6.5.5 of Deep Learning, the book. The material mentioned has a convincing explanation for symbol-to-symbol differentiation but could not say the same for symbol-to-number differentiation.