2
$\begingroup$

I recently came across symbol-to-symbol and symbol-to-number differentiation, out of which symbol to symbol seemed fairly straightforward - the computational graph is extended to include gradient calculations and relationships between gradients.

I have a problem in understanding what exactly symbol-to-number differentiation is. Does it map directly every variable in the backprop to its relevant gradient? If yes, how does it do this without knowing about the rest of the computational graph?

If the question is unclear, to increase context - TensorFlow uses symbol to symbol differentiation whereas torch uses symbol to number (apparently).

Came across this in section 6.5.5 of Deep Learning, the book. The material mentioned has a convincing explanation for symbol-to-symbol differentiation but could not say the same for symbol-to-number differentiation.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ You should link the resources. $\endgroup$
    – user9947
    Apr 7, 2020 at 2:13
  • $\begingroup$ @DuttaA - Assuming you mean where I came across this - Section 6.5.5 of deeplearningbook.org/contents/mlp.html $\endgroup$
    – ashenoy
    Apr 8, 2020 at 8:09
  • $\begingroup$ This seems more of a syntactic problem. In TF you cannot access values arbitrarily, and call to a variable always returns some kind of object (which probably represents the part of graph) whereas in PyTorch the values are directly accessible. $\endgroup$
    – user9947
    Apr 9, 2020 at 9:15

0

Reset to default

You must log in to answer this question.