The math that you need to be comfortable with most deep learning (DL) topics (such as neural networks, gradient descent are back-propagation) is already mentioned in your post, but I will list the main subjects here too.
- Linear algebra (an entire college-level course is necessary; you can start with Khan Academy videos/lessons and you can pick one of Gilbert Strang's books)
- Calculus (same; Kenneth A. Ross' book is a decent one)
- Numerical analysis/algorithms (you need be aware of numerical algorithms, like gradient descent, and concepts like convergence, round-off errors, etc; in fact, gradient descent is the widely used in DL)
- Probability theory (you need to know what a probability distribution, random variable, etc., are)
- Statistics (you don't need to know everything at the beginning, but the more you know the better)
I didn't use this book when I was studying deep learning, but part 1 of this book covers (at least some of) the most important mathematical prerequisites for deep learning, so you could try to read some of the chapters to understand at what point you are. I don't have a favourite book for the last 3 topics listed above.
Check out also the book Mathematics for Machine Learning. I never read it, but it looks like part 1 has many chapters on most important math topics for ML and so DL.
By the way, I don't think that 3 weeks is a lot. You will definitely need more time to learn the mathematical prerequisites for deep learning, but the exact time depends on your specific background.