3
votes
1
vote
Is information gain only calculated using decrease in entropy of a dataset?
Information gain is a very specific measure in information theory, thus the entropy definition... for sure, the decrease in Gini index can be seen as a proxy for it
- 2,632
1
vote
Accepted
How do I add Entropy to a PPO algorithm?
In the regular PPO formulation, which I don't believe is strictly following maximum-entropy RL, only the actor has the entropy term which is employed as a regularizer to favor exploration. (I guess ...
- 3,186
1
vote
What does the product of probabilities raised to own powers used for entropy calculation quantify?
I don't know if $N(X)$ has a name or has any applicability in AI, but I can comment on how this function varies as the $H(X)$ based on your equation
$$N(X) = \dfrac{1}{2^{H(X)}}$$
which looks correct ...
- 41.4k
Only top scored, non community-wiki answers of a minimum length are eligible