0
$\begingroup$

I need to implement a deep neural network [residual neural network (ResNet)] that takes some text as an input [length M x N] and then processes it. Now as far as my understanding goes, ResNet is used for images and not for text. Is there any way I can use it for text? Any article, research paper or video link or any kind of logic how it will work will help.

NOTE:: I don't have any kind of code right now to post. I'm collecting data and then I'll start coding but using ResNet is a must.

Improve this question
$\endgroup$
2
  • $\begingroup$ A couple of questions: 1. Why do you need to use ResNet and are not allowed to use RNNs such as LSTMs or GRUs? 2. ResNet is usually based on CNNs. Does a ResNet MLP also fit your constraints? The problem here is that convolutions don't make much sense for discrete input (i.e. text) 3. How much time do you have for your project? $\endgroup$
    – postnubilaphoebus
    Dec 21, 2022 at 13:19
  • $\begingroup$ To use ResNet is complusary for this part. it's a demand. so can't use LSTM or GRUs. According to requirements they says it fit into this constraints with the text. I have 15 days for this $\endgroup$
    – Python
    Dec 21, 2022 at 13:59

1 Answer 1

Reset to default
2
$\begingroup$

ResNet as a name is defined as a CNN with a specific architecture, but the more general concept of Residual Networks are not necessarily CNNs, but networks that use skip connections.

You could make a Residual MLP, where you have a set of layers that are connected with a skip connection, which would make them residual. An example in Keras functional API would be:

inp = <some input tensor>
x = Dense(64, activation="relu")(inp)
x = Dense(64, activation="relu")(x)
x = BatchNormalization()(x)
block_out = add([x, inp])

The only constraint is that the input to this residual block should have 64 dimensions, for the addition to be possible. If not then concatenation across the last dimension should work too.

Note that any network that uses skip connections is in concept a residual network, for example DenseNet and Transformers fall into this category.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .