4
$\begingroup$

I came across the "reparametrization trick" for the first time in the following paragraph from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.

The Jacobian determinant and variable transformations will become relevant ... when we transform random variables and probability distributions. These transformations are extremely relevant in machine learning in the context of training deep neural networks using the reparametrization trick, also called infinite perturbation analysis.

The trick has been used in the context of neural networks training in the quoted paragraph. But when I search about the reparametrization trick, I found it only or widely in training autoencoders.

In the context of training a traditional deep neural network, is the trick useful?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

The reparameterization trick (also known as the pathwise derivative or infinitesimal perturbation analysis) is a method for calculating the gradient of a function of a random variable. It is used, for example, in variational autoencoders or deterministic policy gradient algorithms.

If you plan on working with models that involve random variables, you definitely need to understand what the reparameterization trick is.

You will also need to understand the other method to calculate gradients for functions of random variables, which is known as the likelihood ratio (also known as the score function or the REINFORCE gradient).

If your definition of a "traditional" neural network does not involve random variables, then such a method is irrelevant.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Note that the re-parametrization trick is used to reduce the variance (i.e. it's a variance reduction technique). The authors of the VAE state this in the VAE paper (if I remember correctly). So, is the re-parametrization really needed or only needed to reduce the variance? Moreover, are you sure that "REINFORCE gradient" is generally a synonym for "likelihood ratio"? Maybe in the context of reinforcement learning and the REINFORCE algorithm, but isn't the likelihood ratio something more general? $\endgroup$
    – nbro
    Nov 19, 2021 at 11:34
  • $\begingroup$ Yes, REINFORCE gradient definitely means the same thing as score function or likelihood ratio (at least when the latter two are talking about gradient estimation). $\endgroup$
    – Taw
    Nov 19, 2021 at 16:22
  • $\begingroup$ The reparameterization trick has lower variance than likelihood ratios. I guess you could call it a variance reduction technique in this sense (assuming that either method can be applied), but typically variance reduction is referring to the use of some sort of control variate. $\endgroup$
    – Taw
    Nov 19, 2021 at 16:27
  • $\begingroup$ I agree with @nbro, REINFORCE gradient is not synonymous with the likelihood ratio. The likelihood ratio is something extremely general from statistics that has been around much longer than REINFORCE. $\endgroup$
    – David
    Dec 19, 2021 at 12:40
  • $\begingroup$ arxiv.org/abs/1506.05254 page 3, nomenclature, "What we call the score function estimator (via [3]) is alternatively called the likelihood ratio estimator [5] and REINFORCE [26]." $\endgroup$
    – Taw
    Dec 19, 2021 at 15:47
0
$\begingroup$

Yes, the reparametrization trick can be useful in the context of variational Bayesian neural networks, although other more effective variance reduction techniques are more commonly used (in particular, the flipout estimator). See this implementation of BNNs that uses Flipout, but TensorFlow Probability, the library used to implement that example, also provides layers that implement the reparametrization trick.

Note that the reparametrization trick is used in the context of variational auto-encoders (VAEs) (so not in the context of deterministic auto-encoders). VAEs and BNNs have a lot in common: both are based on stochastic variational inference (i.e. variational inference combined with stochastic gradient descent). So, whenever you have some sampling or some stochastic operation, the reparametrization trick could turn out to be useful. However, right now, I am only familiar with these two types of models that use it.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .