Can ELMO embeddings be used to find the n most similar sentences?

Question

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a model that can give you the $n$ most similar sentences to an input string?

For reference, gensim provides a doc2vec model that can be trained on a list of strings, then you can use the trained model to infer a vector from some input string. That inferred vector can then be used to find the $n$ most similar vectors.

Could something similar be done, but using ELMO embedding instead?

Any guidance would be greatly appreciated.

Answer 1

I ended up finding this article which does what I'm looking for. Below is the portion of code I adapted for my needs

from sklearn.metrics.pairwise import cosine_similarity

import tensorflow_hub as hub
import tensorflow as tf

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

def elmo_vectors(x):
  embeddings=elmo(x, signature="default", as_dict=True)["elmo"]

  with tf.device('/device:GPU:0'):
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      sess.run(tf.tables_initializer())
      # return average of ELMo features
      return sess.run(tf.reduce_mean(embeddings,1))


corpus=["I'd like an apple juice",
        "An apple a day keeps the doctor away",
         "Eat apple every day",
         "We buy apples every week",
         "We use machine learning for text classification",
         "Text classification is subfield of machine learning"]


elmo_embeddings=[]
print (len(corpus))
for i in range(len(corpus)):
    print (corpus[i])
    elmo_embeddings.append(elmo_vectors([corpus[i]])[0])

print ( elmo_embeddings, len(elmo_embeddings))
print(elmo_embeddings[0].shape)
sims = cosine_similarity(elmo_embeddings, elmo_embeddings)
print(sims)
print(sims.shape)

Answer 2

I'm assuming you are trying to train a network that compares 2 sentences and give how similar they are.

To do that you will need the dataset (the list of sentences) and a corresponding list of 'correct answers' (the exact similarity of the sentences, which I'm assuming you don't have?).

Why do you need to compare them using a neural network though? For python, difflib's sequence matcher would be my suggestion, but I'm sure there are many other libraries out there :)