1
$\begingroup$

In Pix2Pix by Isola et al. they translate images from different pairs of image categories to one another. While most other example applications for the algorithm make sense to me, I'm having difficulties understanding why one would translate facade labels to facade images. As the title says, I already don't see how labeling a facade would help solving any real world problem.

I skimmed the related work of the paper and found little about what "facade parsing" could be used for, except maybe reconstruction from images. Where are facades reconstructed from facade labels? Can anyone tell me other example applications for facade labels and translating them to images?

$\endgroup$
4
  • 1
    $\begingroup$ Does it have to be facade labels? They are just labels into images. The same technique could be used to convert other labels into other images. $\endgroup$
    – user253751
    Aug 30, 2021 at 12:52
  • 1
    $\begingroup$ although maybe facades are useful for a computer game in a city! $\endgroup$
    – user253751
    Aug 30, 2021 at 12:52
  • 1
    $\begingroup$ I guess computer games is the best possible use case. However, I don't know any game that was made using facade labels. The same technique can in fact be used for autonomous driving. Labeling traffic scenes is another example in the paper. But facades? It's probably because labeling facades is a comparably easy computer vision task. $\endgroup$ Aug 30, 2021 at 14:20
  • $\begingroup$ Welcome to SE:AI! This seems like a great question on theory. $\endgroup$
    – DukeZhou
    Aug 31, 2021 at 0:28

1 Answer 1

0
$\begingroup$

It is enlightening to look at how the training data is created. For the purses and shoes they took real purses and shoes and then performed some thing like a county edge on them. They trained the learner using the simpler image to infer texture and color.

There are plenty of folks in the fields like “visual design“ that want to turn a sketch into something more realistic, and the elements of form, and lighting, and space, are important to how they go about doing that. They would use something like this trained pix2pix in their explorations of the transition between the architectural sketch and the real world.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .