I am working on a customized RL environment where each action is represented as a tuple $a = (a_1,a_2,\cdots,a_n)$ such that certain condition must be satisfied for entries of $a$ (for instance, $a_1+a_2+\cdots+a_n \leq \text{constant}$).
I am using the policy gradient method, but I am having some difficulty modeling the underlying probability distribution of actions. Is there any work done in this direction?
For the constraint $a_1+a_2+\cdots+a_n \leq \text{constant}$, I was thinking about generating $n+1$ uniform random variables $U_1,U_2,\cdots,U_n, U$, and set $a_i = \text{constant}\times U \times \frac{U_i}{\sum_{j=1}^n U_j}$. Problem is that the joint density is a bit messy to calculate, which is needed to get the negative log likelihood. I am curious about how such issue is handled in practice.