1
$\begingroup$

Given a neural network model for Covid-19 classification with $C=1$ for positive and $C=0$ for negative enter image description here Let $x_1 = 6$ and $x_2=2$ find

  1. Probability if the patient got Covid-19 $p\left(C=1 | x; w,b\right)$
  2. Probability if the patient didn’t get Covid-19 $p\left(C=0 | x; w,b\right)$
  3. Find the gradient of $CE_{Loss}$

My attempt

For the first problem $$ \begin{aligned} O_1 &= \text{ReLU} \left( b_1 + \sum{x_iw_i} \right) \\ &= \text{ReLU} \left( 0.2 + 6 \cdot 0.3 - 0.2 \cdot 2 \right) \\ &= \text{ReLU} \left( 1.6 \right) \\ &= 1.6 \\ \end{aligned} $$

$$ \begin{aligned} O_2 &= \text{Sig} \left( -0.6 - 0.1 \cdot 2 - 0.2 \cdot 6 \right) \\ &= \text{Sig} \left( -2 \right) \\ &\approx 0.8808 \\ \end{aligned} $$

$$ \begin{aligned} O_3 &= \text{Sig} \left( 0.6 - 0.3 \cdot 0.8808 + 1.6 \cdot 0.5 \right) \\ &\approx 0.75690 \\ \end{aligned} $$

$$ \begin{aligned} p\left(C=1 | x; w,b\right) = O_3 \approx 0.75690 \end{aligned} $$

For the second problem $$ \begin{aligned} p\left(C=0 | x; w,b\right) = 1 - \left(C=1 | x; w,b\right) = 0.2431 \end{aligned} $$

For the third problem

Since it is a lot of things to calculate I'll take $\frac{\delta L}{\delta w_6}$ as example $$ \begin{aligned} \frac{\delta L}{\delta w_6} &= \frac{\delta L}{\delta \hat{y}} \cdot \frac{\delta \hat{y}}{\delta w_6} \\ &= \left(\hat{y}-y\right)O_2\left(O_3 \left(1-O_3\right)\right)\\ &= \left(O_3-y\right)O_2\left(O_3 \left(1-O_3\right)\right) \end{aligned} $$ The answer key says it should be $\left(O_3-y\right)O_2$. Where did I go wrong?

$\endgroup$
1
  • $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Dec 4, 2022 at 17:27

1 Answer 1

3
$\begingroup$

For the first problem your $O_2$ is mistaken and should be corrected as below $$ \begin{aligned} O_2 &= \text{Sig} \left( -0.6 - 0.1 \cdot 2 + 0.2 \cdot 6 \right) \\ &= \text{Sig} \left( 0.4 \right) \\ &\approx 0.5987 \\ \end{aligned} $$

$$ \begin{aligned} O_3 &= \text{Sig} \left( 0.6 - 0.3 \cdot 0.5987 + 1.6 \cdot 0.5 \right) \\ &\approx 0.7721 \\ \end{aligned} $$

$$ \begin{aligned} p\left(C=1 | x; w,b\right) = O_3 \approx 0.7721 \end{aligned} $$

For the second problem thus corrected to: $$ \begin{aligned} p\left(C=0 | x; w,b\right) = 1 - \left(C=1 | x; w,b\right) = 0.2279 \end{aligned} $$

For the third problem since $w_6$ is one of the weights connecting the output layer and we assume $\hat{y}=g(z^{(3)})$, where $z^{(3)}$ is the net input for the output unit and $g$ is its sigmoid transfer function. Then we arrive at: $$ \begin{aligned} \frac{\delta L}{\delta w_6} &= \frac{\delta L}{\delta \hat{y}} \cdot \frac{\delta \hat{y}}{\delta z^{(3)}} \cdot \frac{\delta z^{(3)}}{\delta w_6} \\ \end{aligned} $$

and since $L$ is the binary cross-entropy loss function, we know $\frac{\delta L}{\delta \hat{y}} = \frac{(\hat{y}-y)}{(1-\hat{y})\hat{y}}$. And from above transfer function we know $\frac{\delta \hat{y}}{\delta z^{(3)}}=(1-\hat{y})\hat{y}$, and $\frac{\delta z^{(3)}}{\delta w_6}=O_2$. Thus finally we simplify as $\frac{\delta L}{\delta w_6}=(O_3-y)O_2$

$\endgroup$
2
  • $\begingroup$ Thanks for your response. One more thing, for $\frac{\delta L}{\delta w_4}$ is it $\frac{\delta L}{\delta \hat{y}} \cdot \frac{\delta \hat{y}}{\delta z^{(3)}} \cdot \frac{\delta z^{(3)}}{\delta z^{(2)}} \cdot \frac{ \delta z^{(2)}}{\delta w_4} = (O_3-y)(1-O_2)O_2x_2$? $\endgroup$ Dec 5, 2022 at 15:30
  • $\begingroup$ For the hidden layer it's more complex and you need to further chain $\frac{\delta z^{(3)}}{\delta z^{(2)}}$ into $\frac{\delta z^{(3)}}{\delta O_2} \cdot \frac{\delta O_2}{\delta z^{(2)}}$, and then try to simplify from there. $\endgroup$
    – cinch
    Dec 6, 2022 at 0:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .