5
$\begingroup$

I am working on a problem in which I need to train a neural network to map one or more input images to one or more output images (1 channel for image). Below I report some examples of input&output. In this case I report 1 input and 1 output image, but may need to pass to more inputs and outputs, maybe by encoding this in channels. However, the images are all of this kind, maybe rotated, tralated or changed a bit in shape. (fyi, they are fields defined by fluid dynamics simulations)

I was thinking about CNN, but the standard architecture used for image classification (convolutional layers + fully connected layers) seems not to be the best choice. Instead, I tried using the U-net architecture, composed of compression+decompression convolutional layers. This works quite fine, but maybe there is some other architecture that could be more suited to my problem.

Any suggestion would be appreciated!

input image output image

Improve this question
$\endgroup$
2
  • $\begingroup$ did you figured out what worked well. I am kind of working on the similar problem. If will be great if you can share any ideas. Thank you $\endgroup$
    – parth modi
    Nov 30, 2022 at 10:45
  • $\begingroup$ Something close to UNet has given the best performances $\endgroup$
    – Giulio Ortali
    Dec 1, 2022 at 11:14

1 Answer 1

Reset to default
2
$\begingroup$

Since you have already tried U-Net. You may look into Siamese Networks (with CNNs for images), they are very well known for computing similarity via deep learning. This is a central idea and can be performed with both text and images. As a tip, you may be able to leverage a lot of architecture from U-Net to Siamese.

Hope it helps, Some useful links to start with :

  1. https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04
  2. https://www.aclweb.org/anthology/W16-1617.pdf
Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, thank you for the tip. However I don't see how I could use the siamese architecture in my problem, I have only one image from which I should output another image, I am not comparing anything $\endgroup$
    – Giulio Ortali
    Jan 27, 2020 at 7:59
  • $\begingroup$ It will be great if you can provide more information about your dataset. (train/test). Another suggestion could convert your U-NET to VAE (variational autoencoder). Using this you can generate more images given an image. Let me know if this helps I can edit the answer above. Training setup can be of 2 types with VAE (1)unsupervised and (2)supervised. Usually, its known to be trained in unsupervised fashion but in your case supervised could be better. Again, this raises, why this is different from U-NET?- I think because you can control the variation. But we can observe results empirically. $\endgroup$
    – HARSH NILESH PATHAK
    Jan 28, 2020 at 4:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .