0
$\begingroup$

I'm learning about AutoRegressive Models used on images, and I've studied the training phase, where you model each pixel on the basis of the previous ones using a certain model architecture.

My question is about generating new images (sampling).

I've seen that the sampling is usually done setting manually the value of the first pixel and calculating the following pixels using the model, i.e. for every pixel you want to generate, you take the previous n pixels and give them to the model which outputs the most likely value, where here "likely" is to be intended as the value which is output given the parameters fitted on the training dataset.

But since the model parameters are fixed, and since you set only the first pixel, does this mean that all pixels except the first one are deterministic and hence you can only generate 256 images (256 is the number of possible values of the first pixel in grayscale)?

Thank you!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

if you just take the maximum likelihood color, then probably yes, but you should not...

In other words, assume an distribution, for example a gaussian centered at the predicted color with some variance, and sample from it... this way, you will likely sample a color that is near the one that the model has predicted, but with a bit of noise, which should cause little to no difference visually but will drive the model in different areas of the distribution that it has learned

(not to mention that potentially, you could just sample a floating point number as first pixel, like $128.3$, and the model won't complain)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. What do you mean with "if you take the ML color"? All the pixel following the first one are conditioned on the previous ones, so you choose only the first. Also, the model parameters shouldn't be set so that a slight change in value of a pixel causes the following pixels to change drastically, so even with floating point numbers the range of images you can obtain shouldn't be so wide, it should be approximately like the range of images used for training. Am I missing something? $\endgroup$
    – SuperFluo
    Aug 28 at 8:33
  • $\begingroup$ @SuperFluo say you have to pick the first pixel, and you setup you net with a single linear unit, which says that it should be the color "124.8", now, you can interpret this as the net predicting $\mu$ of a certain Normal Distribution, thus $o \sim N(\mu^\theta;\sigma | x)$... in other words, you can sample a color from $N(124.8, \sigma)$ where sigma can be whatever... if it's 0, then you get a Dirac distribution which implies you just pick the ML color, which is the predicted one... in poor words, you introduce some noise in the process so that you have infinite samples $\endgroup$
    – Alberto
    Aug 31 at 10:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .