I am trying to implement an extractive text summarization model. I am using keras and tensorflow. I have used bert sentence embeddings and the embeddings are fed into an LSTM layer and then to a Dense layer with sigmoid activation function. I have used adam optimizer and binary crossentropy as the loss function. The input to the model are the sentence embeddings.
The training y labels is a 2d-array i.e [array_of_documents[array_of_biniary_labels_foreach_sentence]]
The problem is that during training, I am getting the training accuracy of around 0.22 and loss 0.6.
How can I improve my accuracy for the model?
