1
$\begingroup$

My question may be a bit hard to explain... My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2.

From this action_mean, I create a normal distribution where the network has to learn to adjust the standard deviation. The output of the network is a sample of this normal distribution.

Since the value of action_mean is directly taken from the input, somehow the gradient can't be computed or gives Nones, respectively, because the output of the net is not completely connected with the input.

Would there be a way to link my action_mean with the input value, without changing the input values itself? To describe my problem, I attached a simplified computational graph how tensorboard shows it.

I would be very thankful for any help!

enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ This part "Since the value of action_mean is directly taken from the input, somehow the gradient can't be computed or gives Nones, respectively, because the output of the net is not completely connected with the input." is not clear to me. It's not clear to me why you can't directly connect the output of one NN to the input of another NN. What is this "action mean" supposed to represent for the second network? Are you training both NNs jointly? $\endgroup$
    – nbro ♦
    Nov 20 '20 at 1:16
  • $\begingroup$ @nbro It is only one network and action_mean should represent the mean value of a normal distribution. I can write it in pseudocode for better clarity: def forward(inp1, inp2): x=softmax(layer1(inp1)) dist1 = Canonical(x) index = dist1.sample() action_mean = inp2[index] dist2 = Normal(action_mean, action_std) $\endgroup$
    – Ardi
    Nov 20 '20 at 11:06
  • $\begingroup$ @nbro where action_std is a parameter defined in init. I dont know why but at that point where action_mean is defined, the network seems to be disconnected somehow. Overall, I need a constrained Normal distribution for certain regions and these regions will be provided by the canonical distribution and input2. $\endgroup$
    – Ardi
    Nov 20 '20 at 11:13
  • $\begingroup$ When you say "I dont know why but at that point where action_mean is defined, the network seems to be disconnected somehow. ", what do you mean by "it seems to be disconnected somehow"? Are you getting an error that tells you that you cannot back-propagate? That may be because something is not differentiable in your model (don't know, just guessing). Which library are you using? PyTorch? $\endgroup$
    – nbro ♦
    Nov 20 '20 at 13:07
  • $\begingroup$ @nbro yes indeed, there is an error when back-propagating. I inspected the gradients and some of them are None. The problem lies in the definition of action_mean, which is just a value of input2 (depending on the index) without layers or any other operation in between. So, it cant propagate the gradient back to the input. And yes, I am using PyTorch. I hope you see my problem now... $\endgroup$
    – Ardi
    Nov 21 '20 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.