I am reading a paper implementing a deep deterministic policy gradient algorithm for portfolio management. My question is about a specific neural network implementation they depict in this picture (paper, picture is on page 14).

enter image description here

The first three steps are convolutions. Once they have reduced the initial tensor into a vector, they add that little yellow square entry to the vector, called the cash bias, and then they do a softmax operation.

The paper does not go into any detail about what this bias term could be, they just say that they add this bias before the softmax. This makes me think that perhaps this is a standard step? But I don't know if this is a learnable parameter, or just a scalar constant they concatenate to the vector prior to the softmax.

I have two questions:

1) When they write softmax, is it safe to assume that this is just a softmax function, with no learnable parameters? Or is this meant to depict a fully connected linear layer, with a softmax activation?

2) If it's the latter, then I can interpret the cash bias as being a constant term they concatenate to the vector before the fully connected layer, just to add one more feature for the cash assets. However, if softmax means just a function, then what is this cash bias? It must be a constant that they implement, but I don't see what the use of that would be, how can you pick a constant scalar that you are confident will have the intended impact on the softmax output to bias the network to put some weight on that feature (cash)?

Any comments/interpretations are appreciated!


2 Answers 2


to the cash bias: I think this is simply the money that is still available at time t=50 and has not yet been invested.

  1. Yes, the softmax is just a softmax, applied to the 12 values from the final linear layer.

  2. As far as the "cash bias" goes — I'm not sure if the "cash" part has significance to the authors. But, the "bias" part is standard. The inputs to the last layer are 11-dimensional vectors. But the last layer has 12 parameters: one parameter for each input value, and one additive parameter called the bias.

That is, assuming the input vector is $(x_1, x_2, ..., x_{11})$, then output will be:

$$ y = w_0 + x_1w_1 + x_2w_2 + ... + x_{11}w_{11} $$

The weight $w_0$ is called a bias. If you remove it, then you'll always map the input of all zeros, $(0, ... 0)$ to the output of $0$. This might not be desirable, and might prevent the layer from converging. Adding a bias $w_0$ allows the layer to map any input to any output.

Not that the $w_0$ is not a true constant. It's a trainable parameter like all others.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .