My question may be a bit hard to explain... My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2.
From this action_mean, I create a normal distribution where the network has to learn to adjust the standard deviation. The output of the network is a sample of this normal distribution.
Since the value of action_mean is directly taken from the input, somehow the gradient can't be computed or gives Nones, respectively, because the output of the net is not completely connected with the input.
Would there be a way to link my action_mean with the input value, without changing the input values itself? To describe my problem, I attached a simplified computational graph how tensorboard shows it.
I would be very thankful for any help!
def forward(inp1, inp2): x=softmax(layer1(inp1)) dist1 = Canonical(x) index = dist1.sample() action_mean = inp2[index] dist2 = Normal(action_mean, action_std)
$\endgroup$