# How are partial derivatives calculated in a computational graph?

I am trying to understand how are partial derivatives calculated in a computational graph. I understand reasoning behind computational graphs and I am bold enough to say I understand how they work, at least on high level of understanding.

But what I don't know is how are partial derivatives itself computed or better said, how are they implemented in code.

I have checked few resources like this lecture slides from CS231N, this blog post or this blog post on TowardsDatascience. They explain graphs and how are expressions calculated in graph, but they don't explain how are partial derivatives derived (or I didn't understand from those explanations). For example, blog post from TowardsDatascience says:

Next, we need to calculate the partial derivatives of each connection between operations, represented by the edges. These are the calculations of the partials of each edge:

And then they show image with values of partial derivatives but they newer explain how are these equations actually calculated in implementation of graph.

Yes, okay, I know how to calculate these partial derivatives on paper and then hardcode them in my code, but I don't know how are they actually automatically computed and implemented in code of libraries like Torch or Theano.

Do they have some basic rules implemented in code, like, for example:

$$\frac{\partial (a + b)}{\partial a} = \frac{\partial a}{\partial a} + \frac{\partial b}{\partial a} = 1$$

and then decompose expressions until they reach basic elements/rules or is there another way that libraries like Torch, Theano or TF do it?

Or to put it in another way, if I have this code in Torch:

from torch import Tensor

def element(val):

# Input nodes
i = element(2)
j = element(3)
k = element(5)
l = element(7)

# Middle and output layers
m = i*j
n = m+k
y = n*l

# Calculate the partial derivative
y.backward()

how does Torch know, that is, how does it compute internally that $$\frac{\partial y}{\partial j} = l \cdot 1 \cdot i = l \cdot i$$?