# How to build a neural network that can learn to predict output images?

I am working with a dataset where each input sample is a matrix, and the output corresponding to each input is also a matrix (of shape (400, 10)). The input samples do not have translation invariance. Each output image has shape (16, 16) . The output matrices have translation invariance.

I want to build a neural network which can learn how to predict the output images from the aforementioned data. It seems to me that one needs to think of a neural network here which does regression on the output images to learn. Presently, I am using 1000 data samples for learning in the neural network (input samples and corresponding output images). What is the best way to build a neural network for this task?

Presently, I am using multi-layer perceptron (MLP) with mean square error (MSE) loss for this task. I flatten the input matrices before I feed them into the MLP, and use a standard MLP with multiple hidden layers (4-5) with many hidden units for this task.

While a visual inspection shows that the true and predicted output images are in relatively good agreement for training data, I find a mismatch between true and predicted (reconstructed) output images for validation data. The bottom plot shows pictures of true and predicted (reconstructed) output images for training and validation data for chosen samples.

Presently, I am using training and validation loss curves (with respect to iteration) to measure performance. I want to have a robust metric for comparison which can tell me whether the prediction is a random image or not.

How can I get the model to generalize to the validation set better?

The Python code that I am using for this MLP and the required data can be found here and here respectively.

• Hi. Welcome to AI.SE. Your post is more likely to get responded to if it consists of a single question. Also, implementation questions are off-topic here. This post has the best chance of not being closed as off-topic if you focus on conceptual questions. For more information, see these help pages. – Philip Raeisghasem Apr 4 '19 at 6:02
• Hi Philip, thanks for your suggestion. I have edited the question after your suggestions. I hope it is fine now :) – Siddharth Satpathy Apr 4 '19 at 6:08
• By saying that the input samples do not have translation invariance, do you mean that they are not images? You also say "the output corresponding to each input is also a matrix (of shape (400, 10))", but then you say "Each output image has shape (16, 16)", so what is the difference between the output corresponding to each input sample and the output image (given that they have different dimensions)? I think you should edit your question to clarify this point (at least it is not clear to me). – nbro Apr 4 '19 at 10:18