1
$\begingroup$

In CS231n, I saw the terminology "fully visible belief network", one category of explicit density generative model such as PixelCNN and NADE.

Although I can understand what this terminology wants to say, but I need more explanations in detail:

  1. What exactly does "fully visible" (or "visible") mean?
  2. I have searched about deep belief net, and I believe that it is just a model in category of deep neural network but trained in unsupervised way with energy function. Am I right?
$\endgroup$
1
  • $\begingroup$ deep belief net has little to do with neural network, if not sharing the concept of "neurons" but they are defined in 2 very different ways, though, i have no idea what they mean with fully visible belief network... maybe having no "hidden/latent variables", but it's a pretty loose definition $\endgroup$
    – Alberto
    Sep 1, 2023 at 23:19

1 Answer 1

1
$\begingroup$

It's easier to understand in the context of the lecture. I'm assuming you're referencing this slide in Lecture 12: CS231n Lec 12, slide 16: Taxonomy of Generative Modles: http://cs231n.stanford.edu/2021/slides/2021/lecture_12.pdf

What distinguishes the FVBNs from the other explicit density models are that they have a tractable density: that is, that you can exactly calculate the probability of samples of your dataset (or at least, that's the assumption made by the model). This is what's meant by "fully visible".

Take PixelRNN/CNN, for example. Here, you model images autoregressively, where a given pixel only depends on previous pixels. This allows you to decompose the distribution over many pixels using the chain rule. Look at slide 23 in the lecture for more info. The main idea is that this decomposition allows you to calculate the probability of a sample exactly, making it "fully visible".

The connection to deep belief networks is a bit more iffy in my opinion. The NADE paper explains how the NADE forward pass is similar to an approximation of a RBM. I'm unsure if this extends to other models, however.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .