In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to pass into an RNN encoder. Here's an illustration.

enter image description here

I can easily extract features from either the existing model, like ResNet, VGG, or make a new CNN model easily as they have described in the paper.

For example, let us suppose, I do this

features = keras.applications.ResNet()(images_array) # just hypothetical

How can I convert these images to the grid?? I am supposed to feed the output of the changed grid to an LSTM Encoder as:

keras.layers.LSTM()(grid) # again, hypothetical type

I just want to know what the author means from changing the output in the grid format.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.