These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay.
I am trying to build an AI like that displayed in the diagram: when given a training set of images with their corresponding node maps of face/nose posture, and an image with a missing section (just a gap) with a node map, I would like it to reconstruct the initial image. My thoughts immediately went to GANs for this, but after some searching, the closest I could find were:
- Face recreation without context/not filling gaps, just following pose (DeepFake)
- Filling gaps in images, but with no node reference
- Filling gaps from reference drawings/mappings, but with no way to provide sample images
I would like to hear about any implementations of such an algorithm, if possible optimised for faces, and if none exists, I would like to hear of how I would go about altering the generator of the GAN to work with the context/gap-fill bit (e.g a paper which talks about this idea, but doesn't implement it). Any guidance on the NN that is best for this type of task is also appreciated.