Is it possible to use an RNN to predict a feature that is not an input feature?

Question

I came across RNN's a few minutes ago, which might solve a problem with sequenced data I've had for a while now.

Let's say I have a set of input features, generated every second. Corresponding with these input features, is an output feature (also available every second). One set of input features does not carry enough data to correlate with the output feature, but a sequence of them most definitely does.

I read that RNN's can have node connections along sequences of inputs, which is exactly what I need, but almost all implementations/explanations show prediction of the next word or number in a text-sentence or in a sequence of numbers.

They predict what would be the next input value, the one that completes the sequence. However, in my case, the output feature will only be available during training. During inference, it will only have the input features available.

Is it possible to use RNN in this case? Can it also predict features that are not part of the input features?

Thanks in advance!

Answer 1

Is it possible to use RNN in this case? Can it also predict features that are not part of the input features?

Yes.

No changes are required to a RNN in order to do this. All you need is correctly labelled data mapping a sequence of $x$ to correct $y$ in order to train, and of course a RNN architecture which has input vectors matching shape of $x$ and output vectors matching shape of $y$. The case where $x$ and $y$ are the same data type is just a special case of RNN design, and not a requirement.

You may need to consider some details:

• If the relationship between $x$ and $y$ is complex and non-linear even accounting for accumulated hidden state during the sequence, you may need to add deeper layers. The output of the LSTM can be some vector $h$ and you can add fully-connected layers to help with predicting $y$ from $h$. This, or adding more LSTM layers, is a choice of hyperparameter that you may want to experiment with. Start with a basic LSTM to see how that goes first.

• If you wish to predict a sequence of output features that is either not the same length as the input feature sequence, or logically should come after the whole sequence (think language translation) then this may need a slight change in setup to get best results. For a predict-same-kind sequence you can feed your predicted output value into the next input, but if input and output have different data types, this will not work. Instead, you will need to have some dummy input or other setup for creating sequences of $y$.

In your specific case the second point does not seem to apply, as you want to predict a single $y$ immediately after a sequence of $x$.