What model can solve vector to vector prediction?

Question

I am totally newbie into serial prediction. I am think about which model or AI paradigm can be used to do vector to vector prediction?

For instance, [1,0,1] ^ [0,1,0] = [1,1,1] Another example could be: [1,0,1]^[0,1,0]^[1,1,0]^... = [RESULT]

(I have to say that this example is not proper enough, because obviously we can use a multi-layer model to simply learn the rule of XOR operation with 100% accuracy, but I think it still can indicate what I want. The point is multi-dimensional input and figure out a non-linear rule. Let's say each row vector can be a row of a matrix, I want to find some pattern across each column entries.)

Another better example: [[1,2,3], [4,5,6], [7,8,9]] => [10,3,8]

I know seq2seq model, to some extend can do this job, so as pointer network. But I am just skeptical on my knowledge in this RNN area. And I do need some help.

For the record, I think what I want is different from stock prediction or word prediction or sentence classification. My argue is that these tasks will need an embedding layer to output a one-hot or whatever vectorization and feed the embedding to the model.

But my original input has already been vectors. I did try to feed those vectors as an embedding to the model. But theoretically, it does not make sense to me. And it did not work under my practice.

Please enlighten me.

Yes, I dare to share my code. Please be easy on me.

    # model structure
    model = keras.Sequential()
    model.add(Input(shape=(node_size, node_size)))  # seq, input_dim
    # The output of GRU will be a 3D tensor of shape (batch_size, timesteps, 256)
    model.add(LSTM(1024, return_sequences=True, activation="relu"))
    # The output of SimpleRNN will be a 2D tensor of shape (batch_size, 128)
    model.add(Dropout(0.5))
    model.add(LSTM(node_size, activation="relu"))
    model.add(Dropout(0.5))
    # model.add(Dense(128, activation="relu"))
    # model.add(Dropout(0.1))
    model.add(Dense(node_size, activation="sigmoid"))

    # build model
    opt = keras.optimizers.Adam(learning_rate=0.001)
    model.compile(optimizer=opt, loss="binary_crossentropy", metrics=[BinaryAccuracy()])
    model.summary()
    # A_input = A_input.reshape(1, node_size, node_size)
    # label = label.reshape(1, node_size)

    # train
    model.fit(A_input, label, epochs=200, batch_size=10, validation_split=0.3)
```

Answer 1

Taking the example you provided, you can use a simple RNN or any Seq-Seq architecture. Your input should be of the shape (BATCH_SIZE, NO_OF_TIMESTEPS, FEATURES_AT_EACH_TIMESTEP). So, you basically need to reformat your input as arr = np.array([[1,4,7], [2,5,8], [3,6,9]]) - This assuming that 1,4,7 are representing different set of features at the same time-step (or same dim of diff vectors). Your training output (Y) would be of the shape (BATCH_SIZE, TIME_STEPS) OR arr_y = np.array([[10, 3, 8]]) Then, you can have the model predict the output for the next 3 time-steps (or 3 dimensions of the vector). Note that, in your case you are not really constraining the model to look at the link [TF_simpleRNN][1]. Note that you need to use return_sequences=True` and use a single unit as the output