I've been trying to figure out how to compute the number of Flops in backward pass of ResNet. For forward pass, it seems straightforward: apply the conv filters to the input for each layer. But how does one do the Flops counts for gradient computation and update of all weights during the backward pass?
how to compute Flops in gradient computations for each layer?
what all gradients need to be computed so Flops for each of those can be counted?
How many Flops in computation of gradient for Pool, BatchNorm, and Relu layers?
I understand the chain rule for gradient computation, but having a hard time formulating how it'd apply to weight filters in conv layers of ResNet and how many Flops each of those would take. It'd be very useful to get any comments about method to compute total Flops for Backward pass. Thanks