I am going to design a Neural Net which will be able to break a 5 letter (characters) word into its corresponding syllables (hybrid syllables, I mean it will not strictly adhere to grammatical Syllable rules but will be based on some training sets I provide).

Example : Train -> tra-in

I think of implementing it in terms of some feedforward net as follows :

Input layer ->Hidden layers -> Output layer

There will be 5 input nodes in the form of decimals (1/26 =0.038 for 'A' ; 2/26 = 0.076 for 'B' ......)

The output layer consists of 4 Nodes which corresponds to each gap between two characters in the word.

And fires as follows :

For "TRAIN" (TRA-IN): Input (0.769,0.692,0.038,0.346,0.538) Output(0,0,1,0)

For "BORIC" (BO-RI-C): **Input.... Output (0,1,0,1)

Is it at all possible to implement the Neural Nets in the way I am doing??

And if possible, then how will I decide the number of Hidden layers and Nodes in each layer??

( In the book I am reading, XOR gate problem and its implementation using hidden layer is given . In XOR we could decide the number of Nodes and Hidden Layers required by seeing the Linear Separability of XOR using two lines. But here I think such analysis can't be made.

So how do I proceed?? Or is it a trial and error process?)


I would highly recommend modeling things differently with regard to how letters are presented to the model. While the problem is more natural, perhaps, for a Convolutional or Recurrent Neural Network, there's no problem to try and run this on a feed forward network. However, the way you give letters as input will be very confusing for a network and will make learning very hard. I'd recommend using one hot encoding or even a binary encoding for the letters. If this is for more than playing around I'd try and add some info (encode whether the letter is in "aeiou" in a separate bit).

As for the hidden layers, try playing around a bit. Two systematic approaches are to start very simple and make the model more complicated, or start complicated and make your model simpler (or just normalize a lot). Look at the performance on the training set and on a separate validation set during training. If the model keeps on improving on the training data but starts to deteriorate on the validation data, you're probably over fitting. In this case you should either make the model simpler (fewer nodes, fewer layers) or regularize (start with l2 normalization on the weights). If the data doesn't perform well on the training data, you may wish to make the model more complex.

Once you've tried the feedforward network, really do try CNN or RNNs for this task.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.