I learned how to build recurrent
cnn model for text classification and sketched out my initial implementation. However, I am wondering how to transform recurrent
cnn model for sentence classification. I am curious how can I come up better implementation of recurrent
cnn model for sentence classification task. Here is part of keras solution that I used:
import gensim import numpy as np import string import gensim from gensim.models import Word2Vec from gensim.utils import simple_preprocess from gensim.models.keyedvectors import KeyedVectors word2vec = KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin', limit= 500000,binary=True) embeddings = np.zeros((word2vec.syn0.shape + 1, word2vec.syn0.shape), dtype = "float32") embeddings[:word2vec.syn0.shape] = word2vec.syn0 MAX_TOKENS = word2vec.syn0.shape embedding_dim = word2vec.syn0.shape hidden_dim_1 = 200 hidden_dim_2 = 100 NUM_CLASSES = 10
I want to learn sentence classification task by using recurrent
cnn (RCNN) model. The fact that some people used
RCNN for object recognition problem. And it is not very intuitive for me how to transform same idea to list of short sentences.
Here is the code that I want to make them work for sentence classification task:
document = Input(shape = (None, ), dtype = "int32") left_context = Input(shape = (None, ), dtype = "int32") right_context = Input(shape = (None, ), dtype = "int32") embedder = Embedding(MAX_TOKENS + 1, embedding_dim, weights = [embeddings], trainable = False) doc_embedding = embedder(document) l_embedding = embedder(left_context) r_embedding = embedder(right_context)
continuation of my code
I am struggling to make above code in problem section for sentence classification problem. Can anyone give me possible idea how to make it work for sentences classification?
If there is efficient transformation on above code, I'd like to continue my pipeline as follow to build RCNN model for sentence classification.
forward = LSTM(hidden_dim_1, return_sequences = True)(l_embedding) backward = LSTM(hidden_dim_1, return_sequences = True, go_backwards = True)(r_embedding) backward = Lambda(lambda x: K.reverse(x, axes = 1))(backward) together = concatenate([forward, doc_embedding, backward], axis = 2) semantic = Conv1D(hidden_dim_2, kernel_size = 1, activation = "tanh")(together) pool_rnn = Lambda(lambda x: K.max(x, axis = 1), output_shape = (hidden_dim_2, ))(semantic) model_output = Dense(NUM_CLASSES, input_dim = hidden_dim_2, activation = "softmax")(pool_rnn) model_RCNN = Model(inputs = [document, left_context, right_context], outputs = model_output)
maybe I need to tokenize all sentences and create array for right/left context for each sentences, but I didn't get solid idea on that. Any more thoughts?
how can I realistically create
input matrix for sentences list, right/left context of each sentence? Any workaround to get this done? Any efficient sketch solution to use recurrent
cnn model for sentence classification? Thanks in advance!