I am totally newbie into serial prediction. I am think about which model or AI paradigm can be used to do vector to vector prediction?

For instance, [1,0,1] ^ [0,1,0] = [1,1,1] Another example could be: [1,0,1]^[0,1,0]^[1,1,0]^... = [RESULT]

(I have to say that this example is not proper enough, because obviously we can use a multi-layer model to simply learn the rule of XOR operation with 100% accuracy, but I think it still can indicate what I want. The point is multi-dimensional input and figure out a non-linear rule. Let's say each row vector can be a row of a matrix, I want to find some pattern across each column entries.)

Another better example: [[1,2,3], [4,5,6], [7,8,9]] => [10,3,8]

I know seq2seq model, to some extend can do this job, so as pointer network. But I am just skeptical on my knowledge in this RNN area. And I do need some help.

For the record, I think what I want is different from stock prediction or word prediction or sentence classification. My argue is that these tasks will need an embedding layer to output a one-hot or whatever vectorization and feed the embedding to the model.

But my original input has already been vectors. I did try to feed those vectors as an embedding to the model. But theoretically, it does not make sense to me. And it did not work under my practice.

Please enlighten me.

Yes, I dare to share my code. Please be easy on me.

    # model structure
    model = keras.Sequential()
    model.add(Input(shape=(node_size, node_size)))  # seq, input_dim
    # The output of GRU will be a 3D tensor of shape (batch_size, timesteps, 256)
    model.add(LSTM(1024, return_sequences=True, activation="relu"))
    # The output of SimpleRNN will be a 2D tensor of shape (batch_size, 128)
    model.add(LSTM(node_size, activation="relu"))
    # model.add(Dense(128, activation="relu"))
    # model.add(Dropout(0.1))
    model.add(Dense(node_size, activation="sigmoid"))

    # build model
    opt = keras.optimizers.Adam(learning_rate=0.001)
    model.compile(optimizer=opt, loss="binary_crossentropy", metrics=[BinaryAccuracy()])
    # A_input = A_input.reshape(1, node_size, node_size)
    # label = label.reshape(1, node_size)

    # train
    model.fit(A_input, label, epochs=200, batch_size=10, validation_split=0.3)

1 Answer 1


Taking the example you provided, you can use a simple RNN or any Seq-Seq architecture. Your input should be of the shape (BATCH_SIZE, NO_OF_TIMESTEPS, FEATURES_AT_EACH_TIMESTEP). So, you basically need to reformat your input as arr = np.array([[1,4,7], [2,5,8], [3,6,9]]) - This assuming that 1,4,7 are representing different set of features at the same time-step (or same dim of diff vectors). Your training output (Y) would be of the shape (BATCH_SIZE, TIME_STEPS) OR arr_y = np.array([[10, 3, 8]]) Then, you can have the model predict the output for the next 3 time-steps (or 3 dimensions of the vector). Note that, in your case you are not really constraining the model to look at the link [TF_simpleRNN][1]. Note that you need to use return_sequences=True` and use a single unit as the output

  • $\begingroup$ I did try to use [1,4,7], [2,5,8], [3,6,9] as feature. Basically, lets assume batch_size=1, then my input is (1, 3, 3). One round training is basically '''[3,6,9] -> [2,5,8] -> [1,4,7] -> [MODEL] -> [10,3,8]''' (first three arrows means they get popped into model one after another) I think I did implement in the right way, but it seems like, the model can not efficiently learn a certain rule, even as simple as vector XOR operation $\endgroup$
    – Edee
    Jun 22, 2022 at 4:40
  • $\begingroup$ some of the people say, flat(30, 30) into (1,900) will not impact RNN training, but to me, I think it does effect the RNN training, because flatting (30, 30) -> (1, 900) even change the timestamp and the meaning. I believe mathematically these two has no difference when training, but I just don't feel right. $\endgroup$
    – Edee
    Jun 22, 2022 at 4:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .