How can I read any AI paper?

Question

I have studied linear algebra, probability, and calculus twice. But I don't understand how can I reach the level that I can read any AI paper and understand mathematical notation in it.

What is your strategy when you see the mathematical expression that you can't understand?

For example, in Wasserstein GAN article, there are many advanced mathematical notations. Also, some papers are written by people who have a master's in mathematics, and those people use advanced mathematics in some papers, but I have a CS background.

When you come across this kind of problem, what do you do?

Answer 1

I think the answer depends very much on why you are reading the paper, what are you trying to get out of it? There are plenty of papers that I "read" (or often really just quickly skim through) where I'll definitely not understand all the math. More often than not, this will be because I don't actually care to deeply understand it.

There is plenty of more "practical" research to be done in AI, which definitely doesn't always require a deep understanding of all the math. Intuition can often be enough, at least to get started, for meaningful practical contributions. If this is the sort of research that you're interested in doing, you probably don't need to understand as many of the mathematical parts of AI papers as you do if you're really trying to do research directly in that theoretical area.

Personally, when I write "math-heavy" parts in my own papers (and that will often already be restricted to a rather simple level of math in comparison to the "real theory" ML papers), I always try to make sure to include intuitive, English descriptions of what we're doing around it. Even if you don't immediately understand a full equation, just having the intuitive explanation around it to tell you what it means can be enough for a broad understanding of the paper. Then you only have to dive deep into the details of the equations if -- based on the English text -- you decide that you're actually really interested. So, if there are sufficient, intuitive explanations surrounding the equations, I'd recommend to focus heavily on that first. Not every paper does this though, sometimes there's very little text and very much math, and then this can be difficult.

Even if it turns out that you do have to understand math, you may not have to understand ALL of it right away though. The important parts that I would try to focus on understanding first are:

• A mathematical description of the "problem". This could be an objective function, a metric to be optimised/minimised/maximised, or an existing equation from previous literature that the authors take as a starting point and inspect some detail of in greater detail.
• Mathematical descriptions of the outcomes/results. These could be equations that they actually use in concrete algorithms (see if you can relate them to any pseudocode that may be present), or the final equations stated in theorems / at the end of proofs.

All the complex parts in between are probably less important. Just a vague idea of what the starting point is, and a vague understanding of the final outcome, can be enough to at least know what the paper is about. Then you can decide for yourself whether you really need to know more about the details in between, or if they're maybe not relevant to you / your work / your research.

Answer 2

I think the best way to make reading papers easier is to practice (as in, read lots of papers, try implementing them, etc), and to discuss them with other students/researchers.

Sometimes it's tough to avoid some obscure or really technical math, so you may just need to do extra reading. The Wasserstein metric, for example, is used a lot in ML but I kinda doubt most ML researchers have a good understanding of it. This metric comes from a branch of math called "optimal transportation theory", which is super interesting, but very real-analysis-heavy. If you're really interested in learning about the Wasserstein metric, I recommend Cedric Villani's book "Optimal Transport: Old and New". I also recommend this awesome paper. Nevertheless, learning analysis is likely gonna serve you very well for understanding a wide range of ML papers.

Finally, as a beginning grad student, I have experienced your issue as well. I made a tool to help me with this at this repo, which manages a library of papers you're interested in. It then uses a PageRank algorithm to recommend new papers to you that are commonly referred to by the papers you want to read, with the goal of helping you read up on the foundational "prerequisite" material.

Answer 3

When I read papers in a new domain and when I started reading theoretical ML paper, I faced similar problems. I usually start with the introduction then related work and try to understand all the concepts and related papers cited that are relevant to understanding the paper.

Specifically when it comes to difficult mathematical formulations, as @harwiltz said the more you read about it the easier it gets. There may be a set of papers with concepts that are similar to the paper you are reading but are well-explained I usually read them first (or if it is an important mathematical concept you can find some blogs describing the intuitions/basics behind it).

Answer 4

From my experience (and I've been reading many research papers for a while), it's rare to find a research paper where you fully understand everything in one go, especially if the research paper was published or released recently or a very long time ago (because, back then, maybe people had a different writing style, used a different notation, or something like that), unless you are an expert on the topic, which is probably not the case, unless you are doing serious research on the topic (i.e. you're doing a Ph.D. and beyond; in that case, you probably don't need to ask questions on this site: hopefully, you have a qualified advisor to whom you can ask these questions!), or the paper is really easy and does not contain any formulas.

Of course, if a paper is published, it must contain something novel, so that something novel could be one of the things that you need to spend some time to understand, but the hardest parts of a paper could also easily be the prerequisites (i.e. the concepts that the paper builds upon), because you may not have a very solid knowledge of those topics (as you probably have already experienced).

There are at least three ways to proceed when you are stuck because you don't understand something

1. If you can ignore what you don't understand (i.e. you don't need it for your purposes because e.g. you just need to have a high-level understanding of the topics), ignore it (really!!)
2. If it cannot be ignored (e.g. because you really need to know all the details of the paper because e.g. you need to give a presentation at your university), try to understand what you don't understand by picking up a resource on that topic that you don't understand, then read it; spend the time that you think is opportune (i.e. do not spend 6.5 days to understand a detail of a paper if you only have 7 days to read that paper and prepare a presentation or whatever you need to do)
3. If you can afford it, stop reading that paper and go back to the basics.

In general, learning is not an easy process and, more specifically, reading research papers is not the easiest reading (because research papers are typically concise, i.e. there's a lot of information compression), so do not expect to understand everything of a paper in one go. In fact, the paper How to Read a Paper by S. Keshav, which gives you some guidelines on how to read a paper, tells you to read a paper in three steps. For more details about these three steps, please, read the paper!