1
$\begingroup$

Given a generative model, G, trained on a dataset D. This generative model can be either GAN or Diffusion based. Supposed each sample, x_i, generated by G, can be evaluated by a readily available scoring function, S(x_i).

What are the possible ways to navigate the latent space of G to find generated samples which maximizes S(x_i)?

Here are some of the possible ways I have thought:

  1. Random samplings from G to form a dataset. For each sample, evaluate it with S, then finetune G to maximise S with loss = -S. Keep repeating until G only produces samples with high S values. This is motivated from RLHF but use backprop instead of RL loss.

  2. Fix G, start with a random latent, l_1 to sample x_1 from G. Then slowly find a better x_2 by going into the direction of S, e.g. l_2 = l_1 + alpha * s(x_1) = l_1 + alpha * s( g(l_1)). Is this possible, is it fair to assume P(l, s(1)) is smooth?

  3. Condition the training of G with S, i.e G(X|S(X)). In this way, we can specific any desired score as an input to the generative model.

Are any of these ideas possible? Otherwise, are there better or more simpler approaches?

$\endgroup$

2 Answers 2

1
$\begingroup$

For the first suggestion, are you suggesting doing gradient descent with objective $-S$? If so (and $S$ is differentiable), that's definitely possible. I would suggest looking at the PULSE paper. They do image superresolution over face images by searching through the latent space of a GAN to find high resolution generations that match the given low-resolution image.

The main thing to keep in mind here is that not all areas in the latent space correspond to realistic generations. You probably want areas that correspond to latents that were likely seen during training (e.g., latents with high probability under the prior). The PULSE paper does this by constraining the search space to a hypersphere centered at the origin. $S$ here would be measuring how much the downscaled image deviates.

This paper does something similar. This paper also has an additional perceptual loss term using a VGG model. $S$ here would be measuring the reconstruction loss using both a simple pixel-wise MSE loss and the perceptual loss.

Finally, an early way of generating images from text-prompts was optimizing the input embeddings for an image generator like StyleGAN using CLIP as a loss function.

I'm not too sure about your second and third suggestion, though. The second sounds a bit like gradient descent -- how do you know what "the direction of $S$" is? The third sounds like Conditional GANs, but I'm not aware of work that conditions on the "quality" of the input.

$\endgroup$
1
$\begingroup$

I doubt that you can condition training on a scoring function, if such scoring is not used during training.

Now, assuming your generator is differentiable, it's already been done the "n-step gradient update" on the latent, via gradient descent: $$ z_{t+1} = z_{t} + \alpha \nabla_{z_t} D(G(z_t)) $$ given $D$ the discriminator, $G$ the generator and $z$ the latent noise vector (might be missing a $-$ in the gradient update, depending if the discriminator is trained to predict fakes or reals as $1$)

This approach would be sound if we assume that $D$ is a perfect discriminator, which is hardly the case in real life, thus using it as a score function is very likely to lead to some samples that will indeed fool the discriminator, but are visually worse for humans

If $D$ is not a discriminator, but maybe some other non differentiable scoring function, you can use zero-order optimization to solve estimate the gradient $\nabla_{G(z_t)}S(G(z_t))$, or even directly the gradient with respect to $z_t$

$\endgroup$
2
  • $\begingroup$ Thanks! It makes sense. However for the conditioning method, if assuming we have access to the training, will this be a better method? $\endgroup$ Aug 6, 2023 at 13:45
  • 1
    $\begingroup$ @terenceflow not really, the problem is always that either you have a perfect scoring method, or you will have a bad result... if you check for example AlphaFold, you can consider this the same idea, where they had a model to predict a "good initial guess", and then a very good differentiable model that starting from a good guess, creates a good folding $\endgroup$
    – Alberto
    Aug 6, 2023 at 17:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .