# How to map X to Y for TensorFlow RNN training data

Usually for DNN, I have the training data of matching X (2D) to Y (2D), for example, XOR data:

X = [[0,0],[0,1],[1,0],[1,1]];
Y = [[0],  [1],  [1],  [0]  ];


However, RNN seems strange, I don't get it how to match X to Y, input of RNN layer is 3D and output is 2D (rightclick to open in new tab): https://colab.research.google.com/drive/17IgFuxOYgN5fNO9LKwDijEBkIeWNPas6

import tensorflow as tf;

x = [[[1],[2],[3]], [[4],[5],[6]]];
bsize = 2;
times = 3;

#3d input
input = tf.placeholder(tf.float32, [bsize,times,1]);

cell  = tf.keras.layers.LSTMCell(20);
rnn   = tf.keras.layers.RNN(cell);
hid   = rnn(input);

sess = tf.Session();
init = tf.global_variables_initializer();
sess.run(init);

#results in 2d
print(sess.run(hid, {input:x}));


The example data seen on https://www.tensorflow.org/tutorials/sequences/recurrent are:

 t=0  t=1    t=2  t=3     t=4
[the, brown, fox, is,     quick]
[the, red,   fox, jumped, high]


How to map these data from X (3D input for RNN layer) to Y (2D)? (Y is 2D because RNN layer output is 2D).

• I'm not 100% sure I underststood the question but I'll try to answer it. By default keras is returning a tensor of shape: (batch_size, state_size) which corresponds to the last hidden step for each batch. If you want to return all the hidden states you need to set return_sequences to True, and it will return a 3d tensor with shape: (batch_size, timesteps, output_size). Sep 18 '19 at 11:21
• @razvanc92 actually 2d or 3d is the side matter, the main thing i would like to ask is how to create the training data X, Y for the brown-fox-red-fox example data on tensorflow.org/tutorials/sequences/recurrent Sep 18 '19 at 11:54
• The very first step you need to do, is to embed each work into a fixed size vector. Afterwards from my understanding, at time t=0 the input will be X[:, 0] or [the, the] in your case. The target will be the next time stamp t=1 X[:,1] or [brown, red]. Following this logic you got X already, and for Y it will be X[:, 1:] (all the words from X, pushed by one plus probably a special character to mark the end of the sentence.). Sep 18 '19 at 12:15
• @razvanc92 i know about embedding, but i'm just trying out with class index first Sep 19 '19 at 2:25

I found out how to get 3D output from LSTMCell so that I can matmul with output weights + biases and subtract with expected values:

• Inputs & expecteds should be: placeholder(,[times,batch_size,num_inp]) instead of batch_size first then times. However, tf.keras.layers.LSTM will ask for [batch_size,times,num_inp]
• Use tf.nn.static_rnn with a list of inputs, instead of 1 input

Source code:

import tensorflow as tf;

x = [[[1],[2],[3]],[[4],[5],[6]]];
times = 2;
bsize = 3;

#3d input
inputs = tf.placeholder(tf.float32, [times,bsize,1]);

cell   = tf.nn.rnn_cell.BasicRNNCell(20);
hids,_ = tf.nn.static_rnn(cell,tf.unstack(inputs,times),dtype=tf.float32);

sess = tf.Session();
init = tf.global_variables_initializer();
sess.run(init);

#results in 2d
print(sess.run(hids, {inputs:x}));