0
$\begingroup$

Let's say I have a string "America" and I want to convert it into a number to feed into a machine learning algorithm. If I use two digits for each letter, e.g. A = 01, B = 02 and so on, then the word "America" will be converted to 01XXXXXXXXXX01 (1011). This is a very high number for a long int, and many words longer than "America" are expected.

How can I deal with this problem?

Suggest an algorithm for efficient and meaningful conversions.

$\endgroup$
  • $\begingroup$ If you look at it as a number you treat it like an ordinal scale value. I think you need a hash or maybe a semantic hash. $\endgroup$ – Erhard Dinhobl Nov 22 '16 at 7:58
  • $\begingroup$ Use vectors (list of numbers) to represent the words, not a single number. $\endgroup$ – Ankur Nov 22 '16 at 8:16
  • 1
    $\begingroup$ Could you please clarify how these numbers will be used in the machine learning part? That might affect the answers. $\endgroup$ – Ben N Nov 22 '16 at 14:03
1
$\begingroup$

What are you trying to achieve?

If you need to encode it to some integer use hash table. If you are using something like linear regression or neural network it would be better to use dummy features (one-hot encoding). So for your dictionary of 5 words ("America", "Brazil", "Chile", "Denmark", "Estonia") you get 5 features (x1, x2, x3, x4, x5) which indicate if some word is equal to one in dictionary. So "Brazil" is represented by (0,1,0,0,0), "Germany" is (0,0,0,0,0). Number of features grows with number of words in dictionary making some features practically useless.

If you are using decision trees you don't need to convert string to integer unless specific algorithm asks you to do so. Again, use hash table to do it. In R you can use factor() function.

If you convert your string to integers and use it as single feature ("America" - 123, "Brazil" - 245), algorithm will try to find patterns in it by comparing numbers but may fail to recognize specific countries.

$\endgroup$
1
$\begingroup$

This depends a lot on what you want to achieve, but if you aim to generalise beyond the words encountered in your training data, you should consider using something like word2vec.

In word2vec semantically similar words are represented by similar vectors and what's more, semantic differences translate into geometrical differences. To overuse a standard example: vec(Paris)-vec(France)+vec(Italy)=vec(Rome).

These relationships allow the network to generalise to completely new content.

$\endgroup$
0
$\begingroup$

You shouldn't use a single number for the word, perhaps a number for each letter. Since B isn't the midpoint of A and C, the numbers really shouldn't be 1, 2, 3, etc. One large but effective way of converting is the letter a is 10000000000000000000000000 such that there are 26 digits, and each digit is a letter, so 0000100000... would be E.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.