2
$\begingroup$

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a model that can give you the $n$ most similar sentences to an input string?

For reference, gensim provides a doc2vec model that can be trained on a list of strings, then you can use the trained model to infer a vector from some input string. That inferred vector can then be used to find the $n$ most similar vectors.

Could something similar be done, but using ELMO embedding instead?

Any guidance would be greatly appreciated.

$\endgroup$

2 Answers 2

2
$\begingroup$

I ended up finding this article which does what I'm looking for. Below is the portion of code I adapted for my needs

from sklearn.metrics.pairwise import cosine_similarity

import tensorflow_hub as hub
import tensorflow as tf

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

def elmo_vectors(x):
  embeddings=elmo(x, signature="default", as_dict=True)["elmo"]

  with tf.device('/device:GPU:0'):
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      sess.run(tf.tables_initializer())
      # return average of ELMo features
      return sess.run(tf.reduce_mean(embeddings,1))


corpus=["I'd like an apple juice",
        "An apple a day keeps the doctor away",
         "Eat apple every day",
         "We buy apples every week",
         "We use machine learning for text classification",
         "Text classification is subfield of machine learning"]


elmo_embeddings=[]
print (len(corpus))
for i in range(len(corpus)):
    print (corpus[i])
    elmo_embeddings.append(elmo_vectors([corpus[i]])[0])

print ( elmo_embeddings, len(elmo_embeddings))
print(elmo_embeddings[0].shape)
sims = cosine_similarity(elmo_embeddings, elmo_embeddings)
print(sims)
print(sims.shape)
$\endgroup$
-1
$\begingroup$

I'm assuming you are trying to train a network that compares 2 sentences and give how similar they are.

To do that you will need the dataset (the list of sentences) and a corresponding list of 'correct answers' (the exact similarity of the sentences, which I'm assuming you don't have?).

Why do you need to compare them using a neural network though? For python, difflib's sequence matcher would be my suggestion, but I'm sure there are many other libraries out there :)

$\endgroup$
2
  • $\begingroup$ I appreciate your answer, but I ended up finding my solution elsewhere. Thanks again $\endgroup$ Aug 13, 2019 at 21:24
  • 1
    $\begingroup$ Hi Kenivia! Thanks for trying to contribute to this website. I appreciate that. However, in order to improve your answer, I would suggest you add more details to the sentence Just write a program that does that by comparing words or something. For example, you could mention some existing similarity metrics that could be used for this task. $\endgroup$
    – nbro
    Aug 13, 2019 at 21:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .