# Extracting information from RNA sequence

I am relatively new to machine learning, and I am trying to use a deep neural network to extract some information from sequences of RNA.

A quick overview of RNA: there is both sequence and structure. I am currently expressing the sequence with one-hot encoding (so a sequence of length $$60$$ would be expressed as a $$60 \times 4$$ matrix, with one row for each letter of the sequence, and one column for each possible value of that letter). I am also feeding the 2D structure of the RNA into the network, which is expressed as $$60 \times 60$$ matrix for a sequence of length $$60$$.

I am trying to use these inputs to predict a single continuous value for a given sequence.

Currently, I am using pretty much the exact setup from this tutorial. I chose this architecture because it allows me to separate the inputs (sequence and structure) and have individual layers for them before merging them into a single model. I think this makes more sense than trying to glue the two separate pieces of data together into a single input.

However, the model doesn't seem to be learning anything - validation loss decreases very slightly then plateaus.

If anyone has suggestions, especially someone who has worked with RNA, DNA, or proteins before, I would really, really appreciate it. I am new to this, and I am not sure how to improve my model from here.

def create_mlp(height,width,filters=(16, 16, 32, 32, 64), regress=False):
# initialize the input shape and channel dimension, assuming
# TensorFlow/channels-last ordering
inputShape = (height, width)
chanDim = -1
# define the model input
inputs = Input(shape=inputShape)
# loop over the number of filters
for (i, f) in enumerate(filters):
# if this is the first CONV layer then set the input
# appropriately
if i == 0:
x = inputs
# CONV => RELU => BN => POOL
x = Activation("relu")(x)
x = BatchNormalization(axis=chanDim)(x)
x = MaxPooling1D(pool_size=2)(x)
# flatten the volume, then FC => RELU => BN => DROPOUT
print(x.shape)
x = Flatten()(x)
x = Dense(16)(x)
x = Activation("relu")(x)
x = BatchNormalization(axis=chanDim)(x)
x = Dropout(0.5)(x)
# apply another FC layer, this one to match the number of nodes
# coming out of the MLP
x = Dense(4)(x)
x = Activation("relu")(x)
# check to see if the regression node should be added
if regress:
x = Dense(1, activation="linear")(x)
# construct the CNN
model = Model(inputs, x)
# return the CNN
return model

def create_cnn(width, height, depth, filters=(16, 16, 32, 32, 64), regress=False):
# initialize the input shape and channel dimension, assuming
# TensorFlow/channels-last ordering
inputShape = (height, width, depth)
chanDim = -1
# define the model input
inputs = Input(shape=inputShape)
# loop over the number of filters
for (i, f) in enumerate(filters):
# if this is the first CONV layer then set the input
# appropriately
if i == 0:
x = inputs
# CONV => RELU => BN => POOL
x = Conv2D(f, (3, 3), padding="same")(x)
x = Activation("relu")(x)
x = BatchNormalization(axis=chanDim)(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
# flatten the volume, then FC => RELU => BN => DROPOUT
print(x.shape)
x = Flatten()(x)
x = Dense(16)(x)
x = Activation("relu")(x)
x = BatchNormalization(axis=chanDim)(x)
x = Dropout(0.5)(x)
# apply another FC layer, this one to match the number of nodes
# coming out of the MLP
x = Dense(4)(x)
x = Activation("relu")(x)
# check to see if the regression node should be added
if regress:
x = Dense(1, activation="linear")(x)
# construct the CNN
model = Model(inputs, x)
# return the CNN
return model

mlp = create_mlp(l, 4, regress=False)
cnn = create_cnn(l, l, 1, regress=False)
# create the input to our final set of layers as the *output* of both
# the MLP and CNN
#cnn.output.reshape()
combinedInput = concatenate([mlp.output, cnn.output])

• Hello. Welcome to Artificial Intelligence Stack Exchange. Can you please clarify what you're trying to predict for each sequence/RNA? Moreover, can you clarify why your network is a $60 \times 60$ matrix? This means that if you multiply your input matrix $60 \times 4$ from the right with your first weight matrix you will get another $60 \times 4$ matrix. If this is correct, what do you do then with this new $60 \times 4$ matrix? To get a final number (which is what you're doing, given that you're doing regression), you need to parse this matrix, but how?
– nbro
Feb 10 at 10:21
• Additionally, I would also suggest that you show us the plot of the validation loss, specify how many epochs you trained your model for, what is the size of your training and validation dataset, learning rate, number of parameters of the net, and any other metric you're computing to assess the quality of the predictions/regressions. Finally, please, try to ask a specific question and not just "any suggestions to solve this problem are welcome", then put it in the title so that people immediately understand what you're asking here.
– nbro
Feb 10 at 10:23
• Sorry if these are too many questions at the same time, but sometimes, even though we think that our post/question is clear to us, it's not enough to clear to other people (at least all the possible details that may be helpful to provide an answer may not be available).
– nbro
Feb 10 at 10:26

• Put combined_input through a Dense layer. Also, return Flattened tensors (See Flatten Layer) from both the functions. Feb 16 at 21:08