I am trying to implement a Reinforcement Learning algorithm called DDPG in TensorFlow 2.x on a custom gym environment. I am new to TF. So, I started with the DDPG TF 1.x implementation from pemami4911. My TF 2.x implementation can be found in a google-colab document.

For some reason, the agent would not learn. The output of the actor neural network is always +1 or -1 for the entire episode. My guess is that there is a mistake in one of the following

  1. the Actor and the Critic NN are not properly written
  2. For the Critic NN, there are two inputs. I would like one of them to skip the first layer. Todo this I used Keras.layers.add (see lines 24-40). There might be a mistake in the implementation.
  3. I used Gradient tape to compute the gradient of the critic NN with respect to one of the inputs (Not the parameters). For this I use var = tf.constant(input_actions) and Gradient Tape to compute the gradient using self.tape.gradient(self.prediction, var). This can be found at the line 54-57.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.