# Mutual Information and Link Strength in Bayesian Network

I'm trying to understand how to calculate the strength of every arc in Bayesian Network. I came across this paper, https://smartech.gatech.edu/bitstream/handle/1853/14331/GT-IIC-07-01.pdf?sequence=1&isAllowed=y, but I got lost in the calculation. In particular, how the result in Table 1 (Link Strength true, Link Strength blind, and Mutual Information) derives its value?

Thanks a lot

The $$MI$$ is easy enough. For $$MI(X \to Z)$$, we get $$U([0.5, 0.5]) - 0.5 U([0.9, 0.1]) - 0.5 U([0.1, 0.9])$$. The $$[0.5, 0.5]$$ comes from $$0.5 [0.9, 0.1] + 0.5[0.1, 0.9]$$ -- this is how to calculate $$P(Z)$$ from $$P(Z|X)$$ and $$P(X)$$. For $$MI(X \to Y)$$, we need to marginalize out $$Z$$. This takes a bit of work, but is not too hard. I manually verified all of the reported values in the figure, and they are all correct.
The $$\mathrm{LS}{\mathrm{true}}$$ values are a bit trickier, but still feasible. I managed to get the same values. It is important when calculating $$\mathrm{LS}{\mathrm{true}}(X \to Z)$$ that $$Y$$ is not a parent, so it does not come into it -- which is why the MI gives exactly the same value.
• Thank you very much @Robby, it helps me understand some part. I don't have a strong mathematical background, which makes me struggle to translate the formulas and numbers in the paper. I'm doing a python code for this formula, and still figure out which variables need to hold which numbers. So please apologize. When you mentioned $MI(X \to Z)$ = $U([0.5,0.5])−0.5U([0.9,0.1])−0.5U([0.1,0.9])$, it does use formula (2) $U(Z)-U(Z|X)$, right? There is also a formula $\sum\limits_{x,y}P(x,y) log_{2} (\frac{P(x,y)}{P(x)P(y)})$, and I'm not sure how to derive numbers for $P(x,y), P(x), P(y)$. – qillbel Aug 27 '20 at 4:39
• Yes, I used formula (2) for that. $P(x,y) = P(y | x) \cdot P(x)$. You might want to take an introduction to probability theory, make sure it covers things up to conditional probabilities, entropy and Bayes' theorem; you'll encounter those a lot. – Robby Goetschalckx Aug 27 '20 at 4:48
• Thanks @Robby, I will look at those materials asap. I have quickly tried to find the numbers for: $P(z,x)=0.5; P(x)=0,5; P(z)=0.5$. I got it from $P(z,x)$ = the average of $P(z|x=true)$ and $P(z|x=false)$, which yield 0.5. $P(x) = P(x=true) = 0.5$. $P(z=true)$ = the average of 0.9 and 0.1 = 0.5. But the result is different from the table when I plug in those number in the formula. I believe I calculated those wrongly, but it will help me to also understand the materials that you suggested if I know where my mistakes are. Thanks again Robby and I will update my understanding. – qillbel Aug 27 '20 at 5:17
• If $P(z|x) = 0.5$ and $P(x) = 0.5$, then $P(x, z) = 0.5 \cdot 0.5 = 0.25$. – Robby Goetschalckx Aug 27 '20 at 14:38
• The calculation you did ($P(z=\mathrm{True}|x=\mathrm{True}) P(x=\mathrm{True}) + P(z =\mathrm{True}| x=\mathrm{False})$ gives us $P(z=\mathrm{True})$, not $P(z, x)$. – Robby Goetschalckx Aug 27 '20 at 14:49