Do good approximations produce good gradients?

Question

Let’s say I have a neural net doing classification and I’m doing stochastic gradient descent to train it. If I know that my current approximation is a decent approximation, can I conclude that my gradient is a decent approximation of the gradient of the true classifier everywhere?

Specifically, suppose that I have a true loss function, $f$, and an estimation of it, $f_k$. Is it the case that there exists a $c$ (dependent on $f_k$) such that for all $x$ and $\epsilon > 0$ if $|f(x)-f_k(x)|<\epsilon$ then $|\nabla f(x) - \nabla f_k(x)|<c\epsilon$? This isn’t true for general functions, but it may be true for neural nets. If this exact statement isn’t true, is there something along these lines that is? What if we place some restrictions on the NN?

The goal I have in mind is that I’m trying to figure out how to calculate how long I can use a particular sample to estimate the gradient without the error getting too bad. If I am in a context where resampling is costly, it may be worth reusing the same sample many times as long as I’m not making my error too large. My long-term goal is to come up with a bound on how much error I have if I use the same sample $k$ times, which doesn’t seem to be something in the literature as far as I’ve found.

Answer 1

In general $|f(x) - f_k(x)| \leq \epsilon$ doesn't ensure $|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$. And for neural networks there is no reason to believe it will happen either.

You can also look at Sobolev Training Paper (https://arxiv.org/abs/1706.04859). In particular, note that Sobolev training was better than critic training, which indirectly may indicate approximating function may not be the same as approximating gradient and function. In Sobolev training, the network is trained to match gradient and function whereas in critic training network is trained to match function. They produce quite different results which might give us some hints about the above problem.

In general, if two functions are arbitrary close, they might not be close in gradients.

Edit: (Trying to come up with a negative example) Consider $f(x) = g(x) + \epsilon \sin (\frac {kx} {\epsilon}) $. $g(x)$ is some neural network. Now, we train some another neural network $h(x)$ to fit $f(x)$ and after training we get $h(x) = g(x)$ ($h(x)$ and $g(x)$ have same weights precisely). However, $\nabla f_x =\nabla g_x + k\cos (\frac {kx} {\epsilon})$ is not arbitarariy close $\nabla g_x$.

I hope this example is enough to prove that a neural network that nicely approximates the function may not nicely approximate the gradients and no such result can be proved mathematically rigorously. However, considering the paper in the discussion, you might think for practical purposes it works. However, if you have both function and grad information available that is expected to work better.