Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Specifically, what I am confused about is the term "state". When used in an equation (section 4.5.2), it says:
I know that some system can be in a state, for example, due to the setup of values of different nodes in a graph. But how can a state be described in the case of a neural network and how can the equation above be explained other than that it is the state (or states of several timesteps as in recurrent neural networks) of a neural network?
So the equation that you mentioned is used during the backward pass in which back proppogation is performed in order to make the neural network more accurate. I think you are talking about the state during the forward pass which is completely different. In the forward pass, the neural network is simply run in order to evaluate or it is simply used as a model. The repeating module in long short term memory networks looks like this:
As you can see there are many different parts to this module. There are three main parts. The first is the forget gate layer. This layer tells the cell state or the line running across the stop what to keep. The cell state is kept by this line:
The entire network is based off of manipulating this cell state in order to get accurate results. The equation that you mentioned was related to backprop which is used to train the neural network. This is related to the cell state because it is used to calculate it during the backwards pass. @BlueMoon93 mentioned that this equation has t+1 one in it, but this is because as the recurrent neural network propogates backwards through each module the time goes from high to low. So to conclude, the cell state in an LSTM is one of the vectors that the neural network modifies based on input.