# What are the mathematical prerequisites needed to understand research papers on neural networks? [closed]

I know we have developed some mathematical tools to understand deep neural networks, gradient descent for optimization, and basic calculus. Recently, I encountered arxiv paper that describes higher mathematics for neural networks, such as functional analysis. For example, I remember universal approximation theorem was proved with the Hann-Banach theorem, but I lost the link of that article, so I need to find similar papers or articles to develop my understanding of neural networks mathematically (like with functional analysis, in short, I need to learn more advanced math for research), can you suggest some books or arxiv papers or articles or any other source that describes mathematics for deep neural networks?

• To me, it's not really clear what you're asking. What kind of "mathematical tools" are you really looking for? Are you trying to understand why neural networks can approximate continuous functions? It's not clear what your level of understanding of neural networks is and what you want to understand that you don't understand.
– nbro
Sep 5 '20 at 19:47
• @nbro that is a good point, for example, I want to learn mathematics on the research level. I see many AI labs output brilliant papers and I think they must be using some advanced mathematics, it won't be just random calculation, the should think ahead, so I want to learn mathematics on the research level. is it clear, sorry if It is not clear now, please say and i will go more in details. Sep 5 '20 at 19:49
• Can you please provide an example of paper that you want to understand? There are so many mathematical topics that it's difficult to know them all and so it's difficult to answer your question. However, if you share with us the specific paper or papers that you want to understand, maybe we can provide more details about the type of mathematics that you need to understand those papers. Moreover, I suggest that you edit your post to include these details and also to include a description of your current knowledge of NNs. What do you know? What's your level?
– nbro
Sep 5 '20 at 21:01
• That's difficult to answer because you don't necessarily need big math skills to be an important researcher. It depends on what you research. Also, there are many things in AI, not just neural networks. The short answer to your question is that: you don't need to know all the math to be a good researcher. For example, if you do research on variational inference, you definitely need to know a good dose of calculus, statistics, and probability theory. So, it's difficult to say exactly what you need to learn because there are so many topics and you will almost surely only focus on 1-2 of them.
– nbro
Sep 5 '20 at 21:21
• So, please, ask a more specific question, and you will get more useful answers. We cannot answer your question if you don't tell us what you are interested in. If you say you're interested in becoming an AI researcher, that doesn't mean much, because one AI researcher could be doing research on reinforcement learning, another in expert systems, and another in variational inference. Nobody does research on "AI". Typically, people do research on specific topics or subfields of AI.
– nbro
Sep 5 '20 at 21:29