Please, take a look at Understanding Shannon's Entropy metric for Information. The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign guarantees that information (entropy) is always greater than or equal to $0$.


Is decision tree learning a deterministic algorithm? Given a fixed dataset, does it always produce a tree of a same topology? Generally, yes. Most decision tree learners, like the common ID3 and C4.5/C5.0 algorithms, are deterministic. At each step, the learners consider all possible feature that have not yet been used to split the data, and find the ...


What makes a system deterministic is not the objective of the algorithm or the variability of the data set or lack thereof. It is not the system's academic origin or the label we assign to it or even whether it is predictable that drives whether it is deterministic. A system is deterministic if, given perfectly accurate and comprehensive knowledge of the ...

Only top voted, non community-wiki answers of a minimum length are eligible