I'm trying to understand how to calculate the strength of every arc in a Bayesian Network.

I came across this report Measuring Connection Strengths and Link Strengths in Discrete Bayesian Networks, but I got lost in the calculation.

In particular, how are the values of Link Strength true, Link Strength blind, and Mutual Information computed in Table 1?

enter image description here


The $MI$ is easy enough. For $MI(X \to Z)$, we get $U([0.5, 0.5]) - 0.5 U([0.9, 0.1]) - 0.5 U([0.1, 0.9])$. The $[0.5, 0.5]$ comes from $0.5 [0.9, 0.1] + 0.5[0.1, 0.9]$ -- this is how to calculate $P(Z)$ from $P(Z|X)$ and $P(X)$. For $MI(X \to Y)$, we need to marginalize out $Z$. This takes a bit of work, but is not too hard. I manually verified all of the reported values in the figure, and they are all correct.

The $\mathrm{LS}{\mathrm{true}}$ values are a bit trickier, but still feasible. I managed to get the same values. It is important when calculating $\mathrm{LS}{\mathrm{true}}(X \to Z)$ that $Y$ is not a parent, so it does not come into it -- which is why the MI gives exactly the same value.

The "blind" version took me a bit more effort to implement, but again, I am getting the same values as reported in the table. I used the "simple formula" at the top of page 5.

  • $\begingroup$ Thank you very much @Robby, it helps me understand some part. I don't have a strong mathematical background, which makes me struggle to translate the formulas and numbers in the paper. I'm doing a python code for this formula, and still figure out which variables need to hold which numbers. So please apologize. When you mentioned $MI(X \to Z)$ = $U([0.5,0.5])−0.5U([0.9,0.1])−0.5U([0.1,0.9])$, it does use formula (2) $U(Z)-U(Z|X)$, right? There is also a formula $\sum\limits_{x,y}P(x,y) log_{2} (\frac{P(x,y)}{P(x)P(y)})$, and I'm not sure how to derive numbers for $P(x,y), P(x), P(y)$. $\endgroup$
    – qillbel
    Aug 27 '20 at 4:39
  • $\begingroup$ Yes, I used formula (2) for that. $P(x,y) = P(y | x) \cdot P(x)$. You might want to take an introduction to probability theory, make sure it covers things up to conditional probabilities, entropy and Bayes' theorem; you'll encounter those a lot. $\endgroup$ Aug 27 '20 at 4:48
  • $\begingroup$ Thanks @Robby, I will look at those materials asap. I have quickly tried to find the numbers for: $P(z,x)=0.5; P(x)=0,5; P(z)=0.5$. I got it from $P(z,x)$ = the average of $P(z|x=true)$ and $P(z|x=false)$, which yield 0.5. $P(x) = P(x=true) = 0.5$. $P(z=true)$ = the average of 0.9 and 0.1 = 0.5. But the result is different from the table when I plug in those number in the formula. I believe I calculated those wrongly, but it will help me to also understand the materials that you suggested if I know where my mistakes are. Thanks again Robby and I will update my understanding. $\endgroup$
    – qillbel
    Aug 27 '20 at 5:17
  • $\begingroup$ If $P(z|x) = 0.5$ and $P(x) = 0.5$, then $P(x, z) = 0.5 \cdot 0.5 = 0.25$. $\endgroup$ Aug 27 '20 at 14:38
  • $\begingroup$ The calculation you did ($P(z=\mathrm{True}|x=\mathrm{True}) P(x=\mathrm{True}) + P(z =\mathrm{True}| x=\mathrm{False})$ gives us $P(z=\mathrm{True})$, not $P(z, x)$. $\endgroup$ Aug 27 '20 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.