# What are the rules behind vector product in gradient?

Let's suppose we have calculated the gradient and it came out to be $$f(WX)(1-f(W X))X$$, where $$f()$$ is the sigmoid function, $$W$$ of order $$2\times2$$ is the weight matrix, and $$X$$ is an input vector of order $$2\times 1$$. For ease let $$f(WX)(1-f(W X))=\Bigg[ \begin{array}{c} 0.3 \\ 0.8 \\ \end{array}\Bigg]$$ and $$X=\Bigg[ \begin{array}{c} 1 \\ 0 \\ \end{array}\Bigg]$$. When we multiply these vectors we will multiply them as $$f(WX)(1-f(W X))\times X^T$$ i.e $$\Bigg[ \begin{array}{c} 0.3 \\ 0.8 \\ \end{array}\Bigg]\times[1 \quad0]$$. I do this because I know that we need this gradient to update a $$2\times 2$$ weight matrix, hence, the gradient should have size $$2\times 2$$. But, I don't know the law/rule behind this, if I was just given the values and had no knowledge that we need the solution to update the weight matrix, then, I might have done something like $$[0.3 \quad 0.8]\times\Bigg[ \begin{array}{c} 1 \\ 0 \\ \end{array}\Bigg]$$ which will return a scalar. For a long chain of such operations (multiple derivatives in applying chain rule, resulting in many vectors), how do we know if the multiplication of two vectors should return a vector or matrix (dot or cross product)?

You have $$X$$ which is an $$(2 \times 1)$$ vector, and $$W_1$$ is a $$(1 \times 2 )$$ vector. Their product is a scalar, of which we then take the sigmoid to get our output $$Y_1$$.
The gradient of this w.r.t. $$W_1$$ will be $$f(WX) (1 - f(WX)) X^T$$, which has the appropriate dimensions.