3
$\begingroup$

I have a dataset of images with 9 different classes. However, there are different categories with the same type of associated image and only can be differentiated with an associated matrix in my specific problem.

I want to train a neural network with the images and the associated matrix as inputs. What type of architecture is good to use? Or where can I find bibliography about it?

$\endgroup$
  • $\begingroup$ Hi. What do you mean by "different categories with the same type of associated image". What do you mean by "type of an image"? Also, what do you mean by "associated matrix"? $\endgroup$ – nbro Sep 15 at 11:39
  • $\begingroup$ i have a physic system with an associated hamiltonian (the matrix), this hamiltonian have information about my problem and can be calculated for any time and geometry configuration. The images are png associated with different geometries. I want to classiffy the configurations but there are configuration with the same image and different hamiltonian. $\endgroup$ – Santiago Marín Agudelo Sep 16 at 12:04
0
$\begingroup$

The topic is multimodal neural networks.

Here some repositories that i hope will help me a lot

https://docs.google.com/presentation/d/1z8-GeTXvSuVbcez8R6HOG1Tw_F3A-WETahQdTV38_uc/edit#slide=id.g1ea5aac985_0_892

https://github.com/prml615/prml

https://github.com/husseinmozannar/multimodal-deep-learning-for-disaster-response

https://github.com/guillaume-be/multimodal-avito

Thanks to kbrose, his suggestion led me to this type of architectures.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.