3
$\begingroup$

I've been given an assignment to create a neural network that will suggest a Croatian word for a word given in any other European language (out of those found here). The words are limited to drinks you can find on a bar menu.

I've looked at many NN examples, both simple and complex, but I'm having trouble with understanding how to normalize the input.

For example, words "beer", "birra" and "cervexa" should all translate to "pivo". If I include those 3 in the training set, and after the network has finished training I input the word "bier", the output should be "pivo" again.

I'm not looking for a working solution to this problem, I just need a nudge in the right direction regarding normalization.

$\endgroup$
1
$\begingroup$

The training examples are not mentioned, but, assuming the assignment involves supervised learning, normalization is, in this case, mapping the inputs and outputs to binary values.

Translation, in this simplified context, is a map of strings to strings, which isn't really an AI device by contemporary standards. Since an artificial network is given as a requirement for the solution, the goal is to train the network to act like a hash map.

$$ \Big\{..., \; \big( [\text{"beer", "birra", "cervexa"}]: \text{"pivo"} \big) , \; ...\Big\} $$

$$ \Downarrow $$

$$ \Big\{..., \; \big(\text{"beer" : "pivo"}), (\text{"birra" : "pivo"}), (\text{"cervexa" : "pivo"}\big), \; ...\Big\} $$

Assuming there is a list of pairs $(\mathcal{X}, \mathcal{Y})$, where $\mathcal{X}$ is the word in any of the source languages and $\mathcal{Y}$ is the word in the target language, and the vocabulary of both is known to have word counts of $X$ and $Y$ respectively, one can determine the input and output layer widths (cell counts) $w_1$ and $w_{\ell}$, where $\ell$ is the number of network layers.

$$w_1 = \text{ceiling} ( \log_2{X} )$$ $$w_{\ell} = \text{ceiling} ( \log_2{Y} )$$

The assignment of source words to binary values is arbitrary. You can sort them by language and then by whatever common character encoding used (such as UTF-8) just for debugging purposes. The same is true of target words.

A simple MLP (multi-layer perceptron) with mini-batch training using basic propagation map be appropriate.

$\endgroup$
0
$\begingroup$

So, the normalization as you call it is just encoding the word into some vector of fixed length. After you encode it, you should be able to decode it and match the words. You should google a little, for a seq2seq problems and Encoder/Decoder structure. There are encoder-decoder frameworks out there and lots of resources. You don't care about a language so just pretend that words like "beer" and "birra" have the same meaning in Croatian and in this case, it is "pivo".

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.