Hot answers tagged

3 votes

How to calculate the entropy in the ID3 decision tree algorithm?

Suppose you have data: ...
dynrepsys's user avatar
  • 1,363
1 vote

Is information gain only calculated using decrease in entropy of a dataset?

Information gain is a very specific measure in information theory, thus the entropy definition... for sure, the decrease in Gini index can be seen as a proxy for it
Alberto's user avatar
  • 1,677
1 vote
Accepted

How do I add Entropy to a PPO algorithm?

In the regular PPO formulation, which I don't believe is strictly following maximum-entropy RL, only the actor has the entropy term which is employed as a regularizer to favor exploration. (I guess ...
Luca Anzalone's user avatar
1 vote

What does the product of probabilities raised to own powers used for entropy calculation quantify?

I don't know if $N(X)$ has a name or has any applicability in AI, but I can comment on how this function varies as the $H(X)$ based on your equation $$N(X) = \dfrac{1}{2^{H(X)}}$$ which looks correct ...
nbro's user avatar
  • 40.2k

Only top scored, non community-wiki answers of a minimum length are eligible