My cross entropy loss gradient calculation is wrong according to the answer key

Question

Given a neural network model for Covid-19 classification with $C=1$ for positive and $C=0$ for negative Let $x_1 = 6$ and $x_2=2$ find

1. Probability if the patient got Covid-19 $p\left(C=1 | x; w,b\right)$
2. Probability if the patient didn’t get Covid-19 $p\left(C=0 | x; w,b\right)$
3. Find the gradient of $CE_{Loss}$

My attempt

For the first problem $$ \begin{aligned} O_1 &= \text{ReLU} \left( b_1 + \sum{x_iw_i} \right) \\ &= \text{ReLU} \left( 0.2 + 6 \cdot 0.3 - 0.2 \cdot 2 \right) \\ &= \text{ReLU} \left( 1.6 \right) \\ &= 1.6 \\ \end{aligned} $$

$$ \begin{aligned} O_2 &= \text{Sig} \left( -0.6 - 0.1 \cdot 2 - 0.2 \cdot 6 \right) \\ &= \text{Sig} \left( -2 \right) \\ &\approx 0.8808 \\ \end{aligned} $$

$$ \begin{aligned} O_3 &= \text{Sig} \left( 0.6 - 0.3 \cdot 0.8808 + 1.6 \cdot 0.5 \right) \\ &\approx 0.75690 \\ \end{aligned} $$

$$ \begin{aligned} p\left(C=1 | x; w,b\right) = O_3 \approx 0.75690 \end{aligned} $$

For the second problem $$ \begin{aligned} p\left(C=0 | x; w,b\right) = 1 - \left(C=1 | x; w,b\right) = 0.2431 \end{aligned} $$

For the third problem

Since it is a lot of things to calculate I'll take $\frac{\delta L}{\delta w_6}$ as example $$ \begin{aligned} \frac{\delta L}{\delta w_6} &= \frac{\delta L}{\delta \hat{y}} \cdot \frac{\delta \hat{y}}{\delta w_6} \\ &= \left(\hat{y}-y\right)O_2\left(O_3 \left(1-O_3\right)\right)\\ &= \left(O_3-y\right)O_2\left(O_3 \left(1-O_3\right)\right) \end{aligned} $$ The answer key says it should be $\left(O_3-y\right)O_2$. Where did I go wrong?

Answer 1

For the first problem your $O_2$ is mistaken and should be corrected as below $$ \begin{aligned} O_2 &= \text{Sig} \left( -0.6 - 0.1 \cdot 2 + 0.2 \cdot 6 \right) \\ &= \text{Sig} \left( 0.4 \right) \\ &\approx 0.5987 \\ \end{aligned} $$

$$ \begin{aligned} O_3 &= \text{Sig} \left( 0.6 - 0.3 \cdot 0.5987 + 1.6 \cdot 0.5 \right) \\ &\approx 0.7721 \\ \end{aligned} $$

$$ \begin{aligned} p\left(C=1 | x; w,b\right) = O_3 \approx 0.7721 \end{aligned} $$

For the second problem thus corrected to: $$ \begin{aligned} p\left(C=0 | x; w,b\right) = 1 - \left(C=1 | x; w,b\right) = 0.2279 \end{aligned} $$

For the third problem since $w_6$ is one of the weights connecting the output layer and we assume $\hat{y}=g(z^{(3)})$, where $z^{(3)}$ is the net input for the output unit and $g$ is its sigmoid transfer function. Then we arrive at: $$ \begin{aligned} \frac{\delta L}{\delta w_6} &= \frac{\delta L}{\delta \hat{y}} \cdot \frac{\delta \hat{y}}{\delta z^{(3)}} \cdot \frac{\delta z^{(3)}}{\delta w_6} \\ \end{aligned} $$

and since $L$ is the binary cross-entropy loss function, we know $\frac{\delta L}{\delta \hat{y}} = \frac{(\hat{y}-y)}{(1-\hat{y})\hat{y}}$. And from above transfer function we know $\frac{\delta \hat{y}}{\delta z^{(3)}}=(1-\hat{y})\hat{y}$, and $\frac{\delta z^{(3)}}{\delta w_6}=O_2$. Thus finally we simplify as $\frac{\delta L}{\delta w_6}=(O_3-y)O_2$