2
$\begingroup$

I'm trying to understand how to calculate the strength of every arc in Bayesian Network. I came across this paper, https://smartech.gatech.edu/bitstream/handle/1853/14331/GT-IIC-07-01.pdf?sequence=1&isAllowed=y, but I got lost in the calculation. In particular, how the result in Table 1 (Link Strength true, Link Strength blind, and Mutual Information) derives its value? enter image description here

Thanks a lot

$\endgroup$
0
$\begingroup$

The $MI$ is easy enough. For $MI(X \to Z)$, we get $U([0.5, 0.5]) - 0.5 U([0.9, 0.1]) - 0.5 U([0.1, 0.9])$. The $[0.5, 0.5]$ comes from $0.5 [0.9, 0.1] + 0.5[0.1, 0.9]$ -- this is how to calculate $P(Z)$ from $P(Z|X)$ and $P(X)$. For $MI(X \to Y)$, we need to marginalize out $Z$. This takes a bit of work, but is not too hard. I manually verified all of the reported values in the figure, and they are all correct.

The $\mathrm{LS}{\mathrm{true}}$ values are a bit trickier, but still feasible. I managed to get the same values. It is important when calculating $\mathrm{LS}{\mathrm{true}}(X \to Z)$ that $Y$ is not a parent, so it does not come into it -- which is why the MI gives exactly the same value.

The "blind" version took me a bit more effort to implement, but again, I am getting the same values as reported in the table. I used the "simple formula" at the top of page 5.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you very much @Robby, it helps me understand some part. I don't have a strong mathematical background, which makes me struggle to translate the formulas and numbers in the paper. I'm doing a python code for this formula, and still figure out which variables need to hold which numbers. So please apologize. When you mentioned $MI(X \to Z)$ = $U([0.5,0.5])−0.5U([0.9,0.1])−0.5U([0.1,0.9])$, it does use formula (2) $U(Z)-U(Z|X)$, right? There is also a formula $\sum\limits_{x,y}P(x,y) log_{2} (\frac{P(x,y)}{P(x)P(y)})$, and I'm not sure how to derive numbers for $P(x,y), P(x), P(y)$. $\endgroup$ – qillbel Aug 27 at 4:39
  • $\begingroup$ Yes, I used formula (2) for that. $P(x,y) = P(y | x) \cdot P(x)$. You might want to take an introduction to probability theory, make sure it covers things up to conditional probabilities, entropy and Bayes' theorem; you'll encounter those a lot. $\endgroup$ – Robby Goetschalckx Aug 27 at 4:48
  • $\begingroup$ Thanks @Robby, I will look at those materials asap. I have quickly tried to find the numbers for: $P(z,x)=0.5; P(x)=0,5; P(z)=0.5$. I got it from $P(z,x)$ = the average of $P(z|x=true)$ and $P(z|x=false)$, which yield 0.5. $P(x) = P(x=true) = 0.5$. $P(z=true)$ = the average of 0.9 and 0.1 = 0.5. But the result is different from the table when I plug in those number in the formula. I believe I calculated those wrongly, but it will help me to also understand the materials that you suggested if I know where my mistakes are. Thanks again Robby and I will update my understanding. $\endgroup$ – qillbel Aug 27 at 5:17
  • $\begingroup$ If $P(z|x) = 0.5$ and $P(x) = 0.5$, then $P(x, z) = 0.5 \cdot 0.5 = 0.25$. $\endgroup$ – Robby Goetschalckx Aug 27 at 14:38
  • $\begingroup$ The calculation you did ($P(z=\mathrm{True}|x=\mathrm{True}) P(x=\mathrm{True}) + P(z =\mathrm{True}| x=\mathrm{False})$ gives us $P(z=\mathrm{True})$, not $P(z, x)$. $\endgroup$ – Robby Goetschalckx Aug 27 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.