2
$\begingroup$

I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient calculations and relationships between gradients.

I have a problem in understanding what exactly symbol-to-number differentiation is. Does it map directly every variable in the backprop to its relevant gradient? If yes, how does it do this without knowing about the rest of the computational graph?

If the question is unclear, to increase context - TensorFlow uses symbol to symbol differentiation whereas torch uses symbol to number (apparently).

Came across this in section 6.5.5 of Deep Learning, the book. The material mentioned has a convincing explanation for symbol-to-symbol differentiation but could not say the same for symbol-to-number differentiation.

$\endgroup$
  • 1
    $\begingroup$ You should link the resources. $\endgroup$ – DuttaA Apr 7 at 2:13
  • $\begingroup$ @DuttaA - Assuming you mean where I came across this - Section 6.5.5 of deeplearningbook.org/contents/mlp.html $\endgroup$ – ashenoy Apr 8 at 8:09
  • $\begingroup$ This seems more of a syntactic problem. In TF you cannot access values arbitrarily, and call to a variable always returns some kind of object (which probably represents the part of graph) whereas in PyTorch the values are directly accessible. $\endgroup$ – DuttaA Apr 9 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.