The diagram is nice-ish-looking, and it does lack a convenient list of node type descriptions. There is some description in their Zoo Flashcards page. Although the zoo is not a clean presentation in every respect, it is quite instructive in several ways. These are of particular interest.
- The symbolic shapes and colors visually display the relationship between cell types and the types of networks that employ them, which is quite instructive as an overview.
- The use of the term cell, as opposed to neuron, acknowledges the fundamental difference between current artificial network cells and the amazingly intricate and self-modifying nature of biological neurons.
What is still missing is a concise list of each cell type and its purpose for existing in the contexts of its use. This may be useful.
- Input Cell — Provides a point of input and often uses an identity activation function (the output = the input)
- Back-fed Input Cell — Provides a point where feedback is deliberately introduced
- Noisy Input Cell — Provides a point where chaotic (pseudo-random), thermal, or quantum noise can be introduced
- Hidden Cell — A cell that attenuates (applies a parameterized weight to) the input from all or a select range of activation function outputs from the cells of the immediately prior layer
- Probabilistic Hidden Cell — Applies a radial basis function to the difference between the test case and the cell's mean1
- Spiking Hidden Cell — Employs a spiking model of activation rather than a perceptron model, working toward a more accurate simulation of biological neurons at the cell level
- Output Cell — Provides a point of output and applies the range (min and max) or ordinality (binary, category index)
- Match Input Output Cell — Used in auto-encoders to converge on the identity function from network input to network output, usually through a narrowed data path through which features are extracted after convergence
- Recurrent Cell — Used in RNN designs such as recurrent artificial networks as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
- Memory Cell — Used in LSTM (long short term memory) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
- Different Memory Cell — Used in GRU (gated recurrent unit) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
- Convolution Cell — Passes the result of a convolution kernel multiplied to a relative range of the cells of the immediately previous layer to the next layer
- Pool Cell — Aggregates (usually arithmatically) the results of the cells of a convolution layer
A table of cell types with the features for each might be a next step in presenting the cell types in a comparative summary. The table would particularly helpful if it had sort up and down gadgets in the header row for each column.
Footnotes
[1] Typically $\varphi(\mathbf{x}) = \sum_{i=1}^I a_i e^{\left[ -\beta \left \Vert \mathbf{x} - \mathbf{c}_i \right \Vert ^2 \right]}$ where c is the center value of the cell
[2] These cell types integrate historical influence into the signal path at the cellular level, introducing statefulness beyond the learning parameters and network hyper-parameters, providing the potential of Turing completeness to the network after training.