# During Backpropagation in LSTM, why is the previous output $h_{t-1}$ considered constant w.r.t any $W$ while computing derivative?

I've just started learning LSTM, and some points in the process of calculating the gradients are getting me confused.

Say, for example, we want to compute $$\frac{\partial}{\partial W_i}L$$, where $$L$$ is the loss for one input sequence $$x$$ , $$W_i$$ is the matrix of parameters to learn in the input gate $$i_t$$, and $$h_{t-1}$$ the previous output from last time step as in the equation :

$$i_t = \sigma(W_i h_{t-1} + U_i x(t) + b_i)$$

At some point, during backpropagation, we need to do the "straightforward task", computing $$\frac{\partial}{\partial W_i}i_t$$ which is equal to:

$$\frac{\partial}{\partial W_i}i_t = \sigma(z_{i,t}) (1 - \sigma(z_{i,t})) h_{t-1},$$

where $$z_{i,t} = W_i h_{t-1} + U_i x(t) + b_i$$

Well, my problem is here, I do think that $$h_{t-1}$$ also depends on $$W_i$$ because $$h_{t-1}$$ was computed with $$W_i$$ (and other parameters) during the last time step. So, maybe, some more term should be added to the equation of $$\frac{\partial}{\partial W_i}i_t$$ above.

I hope you can understand my issue.

• It doesn't seem to me that you provide any equation that defines $h_{t-1}$, so am I missing something here? It's been a while since I had to deal with these details, so it's likely I'm missing something. It may be a good idea to link to a reference that shows and explains in detail the equations that you're showing here. For example, where did you get this equation $\frac{\partial}{\partial W_i}i_t = \sigma(z_{i,t}) (1 - \sigma(z_{i,t})) h_{t-1},$ from?
– nbro
Jan 8 at 17:28