1
$\begingroup$

What is the intuition behind the following entropy formula used in the ID3 algorithm?

$$ \text{info}(D) = -\sum_{i=1}^m p_i \log_2(p_i) $$

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Please, take a look at Understanding Shannon's Entropy metric for Information.

The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign guarantees that information (entropy) is always greater than or equal to $0$.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .