I am trying to implement a Reinforcement Learning algorithm called DDPG in TensorFlow 2.x on a custom gym environment. I am new to TF. So, I started with the DDPG TF 1.x implementation from pemami4911. My TF 2.x implementation can be found in a google-colab document.
For some reason, the agent would not learn. The output of the actor neural network is always +1 or -1 for the entire episode. My guess is that there is a mistake in one of the following
- the Actor and the Critic NN are not properly written
- For the Critic NN, there are two inputs. I would like one of them to skip the first layer. Todo this I used
Keras.layers.add(see lines 24-40). There might be a mistake in the implementation.
- I used Gradient tape to compute the gradient of the critic NN with respect to one of the inputs (Not the parameters). For this I use
var = tf.constant(input_actions)and Gradient Tape to compute the gradient using
self.tape.gradient(self.prediction, var). This can be found at the line 54-57.