I have used a FFNN and LSM to perform the same task, namely, to predict the sentence "How are you". The LSM gave me more accurate results than FFNN. However, the LSM did not produce perfect prediction and there are missing letters. More specifically, LSM produced "Hw are yo" and the FFNN predicted "Hnw brf ypu".
What is the difference between a FFNN and a LSM, in terms of architecture and purpose?
Liquid State Machines are used in the field of Neuroscience. What you have used is a variant of LSM called Echo State Network (ESN) used in the field of Machine Learning.
ESN's are pretty simple and superfast compared to normal ML paradigms like Feed Forward NN's or RNN's. ESN's are based on a relatively new paradigm called Reservoir Computing. ESN's are mainly used in Sequential ML problems, and it is used to overcome the problems RNN face like vanisihing gradient, exploding gradient, long training times, etc. The basic idea is that you have a reservoir of Neurons (basically a Neural Net with fixed and non-trainable weights, but it can be varied to complex structures) which will echo an input into some form based on which a simple Linear Regression model (or maybe a complex NN) will be trained.
The basic idea of an ESN is very simple, note it works on time series data and the conventions are similar to a RNN:
The next $x(n)$ is produced as some linear function of non-linear transformation of $x(n-1)$ and $u(n)$ multiplied by their respective weights and $x(n-1)$. $$\tilde x(n) = tanh(a(n))$$ $$ x(n) = (1-\alpha).x(n-1) + \alpha\tilde x(n)$$
The prediction target $y(n)$ is done simply by multiplying $x(n)$ and $u(n)$ with $W_{out}$ which are trainable and the only trainable weights. $$y(n) = W_{out}[1:u(n):x(n)]$$
Now you apply a suitable Loss Function on your predicted $y(n)$ vs actual target $t(n)$ to train weights $W_{out}$
($:$ means concatenation and follows the usual RNN concatenation rules)
This is a very simple overview of ESN's and it works surprisingly well for Sequential prediction tasks. The main idea and intuition behind the working is that the inputs are echoed in the reservoir of untrainable weights which basically means that the input is converted into a certain 'form' due to the random initialisation of weights and this 'form' trains the $W_{out}$ (the first few training $u(n)$ are run through the network without training for the network to assume a starting state where the input has already been echoed in the network and is now coming finally coming out). There are a few mathematical details for the ESN to train and not blow up by huge oscillations and can be found here in this simple intuitive tutorial.
In terms of difference of purpose, a few are:
Although these networks have some very hard to tune hyperparameters, owing to its simple and superfast training one can easily check a lot of hyperparameters. The network also performs surprisingly well and it has resulted in Reservoir Computing being actually researched very actively. Check this very old question too.
