I want to build a Deep Reinforcement Learning Model for Asset allocation.
Background:
I have 7 stock indexes from different markets, and I want to build a policy to produce the action (likes whether to sell or buy index? which index? and how much?) by observing the market informations.
Question 1:
I have two idea for the output of my policy. One is to produce a vector $w$ of length 8, Each element $w_i$ represent the target ratio of the stocks we want to hold (7 stock indexes and 1 cash), so I need to set $w_i>0, $ and $\sum_{i}^{8}w_i=1.$ How to implement? I just let the Activation function in the last layer of neural network to be sigmoid and divide the sum in environment. Is this available? And it's not easy to deal with transaction process if buy and sell fee exist. And the training process slowly when I use the policy gradient.
The two is also produce a vector $w$ of length 8, For each element $w_i$ represent sell percent for stock i when $w_i$ is negative and buy percent of cash when $w_i$ is positive. It can solve the problem I meet in idea one. But I will meet a new question is cash is finite. I need to decide order of buy, in other words, which stock to buy first and buy which one later.
Question 2:
Many papers tell me to produce Distributed parameters by policy then create the action by distribution (like: normal distribution). It makes that more difficult to control the action satisfy the condition above.
And the result whether be unstable if the action is produce by sample.