# Multi-label Classification with non-binary outputs

I am looking to train a dataset that would output a sequence of letters (I'm using this for peptide sequences). Since I have 22 different possibilities of amino acids, I need to output a vector that contains multiple amino acids with varying frequencies. For example, an output would look like [0,1,3,2,0,2,0,3...] (a 22-long vector).

How can I train a neural network to output that type of vector?

• What exactly does your output relate to? I don't understand what 3 or 2 is in your output, could you explain? – Thomas W Jun 9 '17 at 15:00
• @ThomasW Since each peptide sequence has multiple amino acids of the same time, a 3 could represent 3 Ws. For example let's say my peptide sequence is WYTWXTGW, my output vector would have to account for the frequency of each amino acid. – rajkarthikkumar Jun 9 '17 at 15:34
• this doesn't make sense. If you output the frequency of letters, then how do you know the actual sequence? And more importantly, what is your input for this output? – Thomas W Jun 9 '17 at 15:43
• Sorry for the confusion, but the actual sequence is part of the labeled dataset. This is a supervised problem. The input is spectra data that is labeled with a peptide sequence (and thus its corresponding amino acids). The actual data is represented in terms of m/z (a numerical number for spectra data). So if the number is (hypothetically) 4321.32, the peptide sequence could be WYTWXTGW. I want to train a neural network that takes in this information so when it is presented with unlabeled spectra data (after training), it can provide the peptide sequence. – rajkarthikkumar Jun 9 '17 at 15:56
• Oke, the second part helped me understand the problem. If I manage to think of an answer I'll post it. But, I still don't understand the first part. Let's say you only have 2 different aminoacids: A and B. So output would be [x, y]. Say you have output [2,1]. How would you know if the sequence is AAB, ABA or BAA? That's my problem with this :) - i'm not a biological expert, but there are different peptide sequences with the same amount of certain amino acids, right? – Thomas W Jun 9 '17 at 16:04

Technically, you're not really doing classification The outputs you have are not labels, they are real values. So I've got two (possible!) solutions, and you'll have to test them out for yourself:

### Solution 1

Because each output is actually a numerical value, you can normalize them just like you would normalize all other numerical values. So you choose a maximum value that you would expect for the labels (you can also do this per label), and divide the actual value by that value to normalize it.

So if you don't expect more than 10 of the same amino acids in each peptide sequence, then you divide all amounts by 10:

0 becomes 0.0
1 becomes 0.1
2 becomes 0.2
3 becomes 0.3
etc.


### Solution 2

This is more complex, but if you'd give it a try it would surely work. This solution requires recurrent networks. Just like LSTM networks are good for character-by-character text prediction, they will work for your 'peptide sequence language' as well.

This solution also allows an output sequence of any length.

How it works? You have a network with óne output (m/z), and 22 outputs (one-hot encoded amino acids) + 1 more, when this output has the highest value it tells you that the sequence has finished.

You keep on inputting the same value of m/z, until your network tells that the sequence has finished. The network will output different letters every activation based on the previous outputs (the LSTM model has some kind of memory).

This solution is kind of hard to explain, but if you provide me with a small list of peptide sequences (not too long, and not too much), then i'll make a working online example of you.

As promised, I made simple implementation of solution 2. However, I think you should go for solution 1 anyways. I didn't have any real data, so my model is basically overfitting, but it basically works: https://jsfiddle.net/ovpkL2xx/2/ (might take some time to run)