To understand the inner workings of neural networks, a fair amount of mathematical concepts is required. Backpropagation alone is a challenging technique if you are not fluent in calculating local gradients. And that's just the start of the journey.
But the more I study neural networks, the more I get the impression that all those difficult mathematical concepts are only required if you are doing actual research in neural networks or want to know what's happening under the hood. If you "just" want to implement an AI utilizing a neural networks, there are several high level programming frameworks and libraries readily available including model zoos for state of the art neural networks (e.g. VGG, GoogLeNet and ResNet), that can be used.
So my question is, does a developer require a deep understanding of all the details nowadays, or have we reached a level, where frameworks take care of those details for us?