1
$\begingroup$

enter image description here

In the above diagram, the shape of some of the matrices can be seen in the yellow highlight. For instance:

The hidden state at timestep t-1 ($h_{t-1}$) has shape $(na, m)$

The input data at timestep t ($x_{t}$) has shape $(nx, m)$

$Z_{t}$ has shape $(na+nx, m)$ since the hidden state and input data are concatenated in LSTMs.

$W_{c}$ has shape $(na, na+nx)$

$W_{c}$ • $Z_{t}$ has shape $(na, m)$ = $i_{t}$

$W_{i}$ • $Z_{t}$ has shape $(na, m)$ = $ĉ_{t}$

When working through the network to the point $i_{t}$ and $ĉ_{t}$, how can these two be dot producted when the multiplication is not of the form (m x n)(n x p) as per the matrix multiplication definition?:

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Turns out the reason is because, for places where a dot is shown in the image above, they're actually element-wise multiplications, not dot products. A lot of sources use an X or . to denote multiplication, but don't clearly indicate they mean element-wise multiplication.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .