Tag Info

How to calculate the entropy in the ID3 decision tree algorithm?

Suppose you have data: ...
• 1,345
1 vote
Accepted

What is the relationship between entropy in thermodynamics and entropy in information theory?

here are the link to the similar question: Is information entropy the same as thermodynamic entropy? - asked on physics stackexchange. Entropy in information theory is directly analogous to the ...
• 26
1 vote

What does the product of probabilities raised to own powers used for entropy calculation quantify?

I don't know if $N(X)$ has a name or has any applicability in AI, but I can comment on how this function varies as the $H(X)$ based on your equation $$N(X) = \dfrac{1}{2^{H(X)}}$$ which looks correct ...
• 35.6k

Only top scored, non community-wiki answers of a minimum length are eligible