5
$\begingroup$

I am trying to build a neural network that takes in a single string, ex: "dog" as an input, and outputs 50 or so related hashtags such as, "#pug, #dogsarelife, #realbff".

I have thought of using a classifier, but because there is going to be millions of hashtags to choose the optimal one from, and millions of possible words from the english dictionary, it is virtually impossible to search up the probability of each

It is going to be learning information from analyzing twitter posts' text, and its hashtags, and find which hashtags goes with what specific words.

$\endgroup$
  • 1
    $\begingroup$ won't simple logic work better? like if the string contains a certain substring then it must refer to a dog...coz i think its impossible for a machine to learn what words might correspond to a dog without seeing it before..like if someone says pug in some other language then human brain has no way of identifying its a dog...its my opinion although...and logical operations will be much faster $\endgroup$ – DuttaA Jan 22 '18 at 11:24
  • 1
    $\begingroup$ The problem with that is that because I am trying to analyze twitter posts, with hashtags like #follow4follow which are totally unrelated with dogs, I would need a neural network to carry out the decisions. Besides its not going to be only dogs, cats, cars, mountains, calendars, you name it. Basically it is supposed to work like a translator neural network, translating between "strings" and possible #hashtags @DuttaA $\endgroup$ – Tamim Azmain Jan 22 '18 at 11:45
  • 1
    $\begingroup$ That's a NLP problem according to me..and no satisfactory solution exists to NLP..but you might have a case specific solution from experts..lets see $\endgroup$ – DuttaA Jan 22 '18 at 11:55
1
$\begingroup$

Here is a good approach to achieve the task you want:

Step 1- Compute the Vector representation (i.e embeddings) of all the words you want to include. There are many algorithms out there to achieve this task.

Step 1- Compute Embeddings

Step 2- Choose the #words corresponding to your input word (e.g dog) by applying K-Nearest Neighbors (KNN) or similar algorithms. You basically compute the distances using the embeddings.

Step 2 - Apply KNN

Steps Detailed:

Step 1-

In NLP we represent human language as a vector of values instead of a set of characters in order to process it. To do so there are 3 approaches in the literature:

- Word Level Embeddings: Represent each word as a vector of values. Algorithms: Word2Vec by Google (paper), fastText by Facebook, GloVe by Stanford University (paper) ...

- Character Level Embeddings: Represent each character as a vector of values. Algorithms: ELMo (paper) ...

- Sentence Level Embeddings: Represent a sentence as a vector of values. Algorithms: Universal Sentence Encoder by Google (paper) ...

In your case I suggest to use GloVe or ElMo if you have only words and Universal Sentence Encoder if you have words and sentences. . Compute all your words embeddings and move to the next step.

Step 2-

Now that you have your embeddings, compute the distances between all your words (use Euclidian, Minkowski or any other distance). Notice that the computation may take some time but will only be executed once.

Now each time you have a word (e.g dog) you apply the KNN algorithm using the computed distances and you will get the most related words to this word.

Note: No need to compute distances and apply KNN if you use Universal Sentence Encoder as the similarity is easily computed using a dot product of the embeddings. See my quick implementation example here for details.

$\endgroup$
0
$\begingroup$

You can try using Mallet (which uses Gibs Sampling) or gensim LDA (which uses drichlet priors) to model the problem as topics (hashtags) in different documents (tweets). https://towardsdatascience.com/topic-modeling-and-latent-dirichlet-allocation-in-python-9bf156893c24 has a nice example.

$\endgroup$
0
$\begingroup$

I think your intuition about a classifier being the wrong approach is a good one. This looks like a great use-case for word vectors, a "self-supervised" learning technique that maps tokens (e.g. "dog") to vectors (which might have anywhere between 50 - 500 dimensions). Facebook open-sourced a particular excellent tool for training word vectors called FastText; you could use this to embed tokens and hashtags alike into a word embedding space. You should find that the vector for "dog" ends "close to" (small cosine distance). Given a word, you can easily look up its vector (after training on your corpus, of course), but how to find other vectors that are close to it? If you want to do better than "brute force" and you need to check against a large number of (vectors for) hashtags, you should consider using Facebook's excellent FAISS library for fast similarity search to find the closest hashtags.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.