0
$\begingroup$

I have some familiarity with the regular Tensorflow library and have been able to create a number of working models with it. There are more than enough resources out there to get up and running and answer most questions on the standard library.

But I recently came across the video on some high-level capabilities of the Tensorflow Probability library, TensorFlow Probability: Learning with confidence (TF Dev Summit '19), and I would like to learn it.

The issue is that there are very few resources out there on TFP and given my lack of a formal background in math/statistics, I find myself aimlessly googling to get a grasp of what's going on in the docs. I'm more than willing to invest the time needed, but I just need to know where I can start in terms of resources I can access online. Specifically, I'm looking to get the necessary domain knowledge needed to work with the library given the lack of courses/tutorials on the library itself.

Improve this question
$\endgroup$
4
  • $\begingroup$ I'm sorry. I believe that this question is off-topic here and primarily opinion-based, so I should close it. Can you ask a question that is objectively answerable? I am familiar with TFP. If you want, I can create a chat room for discussion. $\endgroup$ – nbro♦ Mar 8 '20 at 22:01
  • $\begingroup$ Could I edit the question to give it more general value? I realize it's very open-ended but wasn't sure how to avoid that. Really I'm just looking for like a list of topics that could be considered prerequisites for working with the library. $\endgroup$ – SuperCodeBrah Mar 8 '20 at 22:10
  • $\begingroup$ Have you already followed any of their tutorials/articles? Anyway, are you asking for the theoretical prerequisites or for resources? $\endgroup$ – nbro♦ Mar 8 '20 at 22:12
  • $\begingroup$ Thanks for your edit. Resources would be great, but high level topics would be a start. On the tutorials, I'm mainly interested in getting a Gaussian mixture model to work, but even that tutorial is a little dense for what I currently know. Before that, I tried a few implementations but hit one issue or another and feel like I just need a better grasp on the fundamentals. I can probably brute force the tutorials (i.e., repetitive googling), but just knowing which areas to investigate would probably help me find courses/tutorials that lay out the broad ideas in a more rational manner. $\endgroup$ – SuperCodeBrah Mar 8 '20 at 22:33
2
$\begingroup$

Although this question is slightly primarily opinion-based and too broad (and I will probably close it as such) and a good answer will necessarily depend on your background, I will list some of the main theoretical prerequisites that everyone should ideally be familiar with before diving into TensorFlow Probability (TFP).

I am familiar with TFP, given that I've been using it for a project, but I've not used all of its functionalities, such as the bijectors. I've only used the Bayesian layers, distributions, etc., so I will try to give an answer based on my experience.

You definitely need to be familiar with the basic concepts of probability theory, such as distributions, random variables, expectations, etc. A full university-level course in probability and statistics would definitely be helpful!

If you want to use Bayesian layers, you also need to be familiar with Bayesian neural networks (BNNs). To understand BNNs, you need to understand the basic concepts of Bayesian inference (BI), given that most BNNs are based on BI. If you are already familiar with variational auto-encoders (VAEs), then it will be easier to understand BNNs. To understand BNNs, you should start reading the paper Weight Uncertainty in Neural Networks. If this paper is difficult to follow, you should first read the paper Variational Inference: A Review for Statisticians and then it may also be useful to read and be familiar with the VAE paper. Therefore, you will need to be familiar with concepts such as Kullback–Leibler divergence and Monte Carlo sampling.

The blog post Regression with Probabilistic Layers in TensorFlow Probability is the first blog post you should read. If you don't understand it, then it probably means you need to learn its prerequisites (i.e. you need to be familiar with at least linear regression).

TFP also provides several implementations of other more advanced concepts such as BNNs. The easiest example to follow is probably the logistic regression example. LR is relatively easy to follow compared to the topics I mentioned above, but, of course, you need to be familiar with logistic regression.

Of course, I cannot list all the theoretical prerequisites (and that's why your post is too broad!) and there are definitely others, but this is a start. Bear in mind that these are not trivial topics, so it is normal to get stuck.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I made a probabilistic determination and marked this as the accepted answer. I looked at the blog post and it was mostly digestible (it was very similar to the topics in and actually links to the video I posted), but I will check out other TFP posts on the Tensorflow blog (hadn't actually looked at the blog before) as well as the papers and links you referenced. Thanks for bearing with me! $\endgroup$ – SuperCodeBrah Mar 8 '20 at 23:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.