I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the next layer. How do I approach this?

The inputs would be an array of documents and each document containing an array of sentences.

The input to the model itself is a list of sentences where it will return a list of embeddings.

This is what I've tried but couldn't solve the errors:

def get_embeddings(input_data):

    input_embed = []
    for doc in input_data:
      doc = tf.unstack(doc)
      doc_arr = asarray(doc)
      doc = [el.decode('UTF-8') for el in doc_arr]
      doc = list(doc)
      assert(type(doc)== list)

      new_doc = []
      for sent in doc:
        sent = tf.unstack(sent)
        assert(type(sent)== str)

      embedding= model.encode(new_doc)  # Accepts lists of strings to return BERT sentence embeddings

    return tf.convert_to_tensor(input_embed, dtype=float)

sentences = tf.keras.layers.Input(shape=(3,5)) #test shape
sent_embed = tf.keras.layers.Lambda(get_embeddings)

x = sent_embed(sentences)



1 Answer 1


I think you should use Keras embedding layer. It will be too easier than what you are doing.


  • Create Embedding Matrix
  • add matrix to embedding layer while building model.

You will find detailed article



You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .