# Can ELMO embeddings be used to find the n most similar sentences?

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a model that can give you the $$n$$ most similar sentences to an input string?

For reference, gensim provides a doc2vec model that can be trained on a list of strings, then you can use the trained model to infer a vector from some input string. That inferred vector can then be used to find the $$n$$ most similar vectors.

Could something similar be done, but using ELMO embedding instead?

Any guidance would be greatly appreciated.

I ended up finding this article which does what I'm looking for. Below is the portion of code I adapted for my needs

from sklearn.metrics.pairwise import cosine_similarity

import tensorflow_hub as hub
import tensorflow as tf

def elmo_vectors(x):
embeddings=elmo(x, signature="default", as_dict=True)["elmo"]

with tf.device('/device:GPU:0'):
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())
# return average of ELMo features
return sess.run(tf.reduce_mean(embeddings,1))

corpus=["I'd like an apple juice",
"An apple a day keeps the doctor away",
"Eat apple every day",
"We use machine learning for text classification",
"Text classification is subfield of machine learning"]

elmo_embeddings=[]
print (len(corpus))
for i in range(len(corpus)):
print (corpus[i])
elmo_embeddings.append(elmo_vectors([corpus[i]])[0])

print ( elmo_embeddings, len(elmo_embeddings))
print(elmo_embeddings[0].shape)
sims = cosine_similarity(elmo_embeddings, elmo_embeddings)
print(sims)
print(sims.shape)


I'm assuming you are trying to train a network that compares 2 sentences and give how similar they are.

To do that you will need the dataset (the list of sentences) and a corresponding list of 'correct answers' (the exact similarity of the sentences, which I'm assuming you don't have?).

Why do you need to compare them using a neural network though? For python, difflib's sequence matcher would be my suggestion, but I'm sure there are many other libraries out there :)

• I appreciate your answer, but I ended up finding my solution elsewhere. Thanks again – brokentransform Aug 13 at 21:24
• Hi Kenivia! Thanks for trying to contribute to this website. I appreciate that. However, in order to improve your answer, I would suggest you add more details to the sentence Just write a program that does that by comparing words or something. For example, you could mention some existing similarity metrics that could be used for this task. – nbro Aug 13 at 21:53