I'm trying to create a deep learning network to classify news article based on the text and associated image. The idea comes from a novel use of GANs to classify based on generated data.
My approach was to use Tensorflow to generate word embeddings in the article, and then tranform the images into records - https://github.com/openai/improved-gan/blob/master/imagenet/convert_imagenet_to_records.py. This second component would also contain the label.
- Is it wise to combine both modes into one neural net, or classify separately?
I'm also trying to work out how to concatenate the two tensors in Tensorflow. Can anyone give a steer.