1
$\begingroup$

Are there some known neural networks that, given an input image, can generate a similar image, with the same topic?

Example: input = a photo of a cat on a green table, output = a generated photo of another cat on another green table.

Example 2: input = a portrait of a man with glasses and a beard, output = a portrait of a generated person with similar glasses / beard (see "ThisPersonDoesNotExist").

I imagine it is possible with a GAN, but more precisely which kind of architecture?

$\endgroup$

1 Answer 1

1
$\begingroup$

The closest literature to what you're suggesting is indeed related to GANs, specifically to arithmetic performed on the latent space learned by generators.

Check Unsupervised Representation Learning with deep convolutional generative adversarial networks.

The idea suggested in the paper is to "tune" specific characteristic of the generated images by discarding specific filters learned by the generator. For example, by removing filters related to specific objects like windows, we could generate images of rooms without windows (see second image below, top row: images generated with all filters, bottom row: images generated without window filters).

enter image description here top row: images generated with all filters, bottom row: images generated without window filters

Other papers applied and refined the same idea, showing that you can tune characteristic of faces like age/glasses and so on.

enter image description here

$\endgroup$

You must log in to answer this question.