I have the following program for my neural network:
n_steps = 9
n_inputs = 36
n_neurons = 50
n_outputs = 1
n_layers = 2
learning_rate = 0.0001
batch_size =100
n_epochs = 1000#200
train_set_size = 1000
test_set_size = 1000
tf.reset_default_graph()
X = tf.placeholder(tf.float32, [None, n_steps, n_inputs],name="input")
y = tf.placeholder(tf.float32, [None, n_outputs],name="output")
layers = [tf.contrib.rnn.LSTMCell(num_units=n_neurons,activation=tf.nn.relu6, use_peepholes = True,name="layer"+str(layer))
for layer in range(n_layers)] layers.append(tf.contrib.rnn.LSTMCell(num_units=n_neurons,activation=tf.nn.relu6, use_peepholes = True,name="layer"+str(layer)))
multi_layer_cell = tf.contrib.rnn.MultiRNNCell(layers)
rnn_outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)
stacked_rnn_outputs = tf.reshape(rnn_outputs, [-1, n_neurons])
stacked_outputs = tf.layers.dense(stacked_rnn_outputs, n_outputs)
outputs = tf.reshape(stacked_outputs, [-1, n_steps, n_outputs])
outputs = outputs[:,n_steps-1,:]
I want to know whether my network is fully connected or not?
When I try to see the variables, I see:
multi_layer_cell.weights
The output is:
[<tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/kernel:0' shape=(86, 200) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/bias:0' shape=(200,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/w_f_diag:0' shape=(50,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/w_i_diag:0' shape=(50,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_0/layer0/w_o_diag:0' shape=(50,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/kernel:0' shape=(100, 200) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/bias:0' shape=(200,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/w_f_diag:0' shape=(50,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/w_i_diag:0' shape=(50,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/layer1/w_o_diag:0' shape=(50,) dtype=float32_ref>]
I didn't understood whether each layer is getting the complete inputs or not.
I want to know whether the following figure is correct for the above code:
If this is not then what is the figure for the network? Please let me know.