Mapping input vectors of variable length to output vectors of variable lengths with dummy variables

Question

I have a general question about supervised ANNs that map inputs to outputs. It is possible to vary the length of the input and output vectors by inserting some dummy variables that will not be considered in the mapping (or will be mapped to other dummy variables). So basically the mapping should look like this (v: value, d: dummy)

Input vector 1 $[v,v,v,v,v] \rightarrow$ Output vector 1 $[v,v,v,v,v]$

Input vector 2 $[v,v,v,v,v]\rightarrow$ Output vector 2 $[v,v,v,v,v]$

Input vector 3 $[v,v,v,d,d] \rightarrow$ Output vector 3 $[v,v,v,d,d]$

Input vector 4 $[v,v,d,d,d] \rightarrow$ Output vector 4 $[v,v,d,d,d]$

Input vector 5 $[v,d,d,d,d] \rightarrow$ Output vector 5 $[v,d,d,d,d]$

The input and output vectors have a length of 5 with 5 values. However, sometimes only a vector of size e.g. 3 (which is basically a vector of length 5 with 2 dummy variables) should be mapped to an output vector of length 3. So after training the ANN should know that if it for example gets an input vector of length 3 it should produce an output vector of length 3.

Is something like this generally possible with ANNs or other machine learning approaches? If so, what type of ANN or machine learning approach can be used for this? I'll appreciate every comment.

Reminder: Can anybody give me more insights into this?

Answer 1

This should be possible but I've never seen it done in practice. Whether or not this will even actually work is unclear to me and will be highly dependent both on your training data and choice of loss. I'd take a step back and look into the literature to see if you can't find a more established approach to your problem, perhaps with RNNs. That being said, I believe the following should do what you're asking.

Consider network $N$ to be a dense neural net with $k$ layers, $N_i$ to be the $i$th layer of $N$, $L$ to be the max length of the input, and $V$ to be the number of terms in the input (the number of $v$s). To accomplish what you want in the above scenario, you can add three additional layers to $N$, $N_{k+1}$ $N_{k+2}$, and $N_{k+3}$:

$N_{k+1}$ is a simple dense layer that has $L$ neurons and takes as input the output of $N_k$. This layer can be skipped if layer $N_k$ already has $L$ neurons.

$N_{k+2}$ takes as input the output $N_{k+1}$ and takes the Hadamard product (elementwise multiplication) of it with a second input, a binary vector of length $L$ with a prefix of $V$ $1$s and suffix of $L-V$ $0$s. For example if $L=5$ and $V=2$, you would supply as the second input the vector $[1, 1, 0, 0, 0]$, which effectively "zeroes out" the third, fourth, and fifth positions.

$N_{k+3}$ is your new output layer, which also has $L$ neurons. $N$ can now be trained and, given the target data is in the proper format to achieve this, it should output results like in your question.