2
$\begingroup$

I wonder if researchers tried to understand how LSTMs work by analyzing the dynamics of simple LSTM (e.g. with 2 units)? For example how the hidden state evolves depending on the properties of weight matrices.

It seems like a very natural thing to try (especially because it is easy to draw hidden states with 2 units on a 2D plane). However, I haven't found any papers that would play with such a toy example. Are there such papers at all? Or with such a simple example, it is impossible to get some understanding because of its over-simplicity? (which I doubt because even logistic maps generate very complicated behavior)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.