4
$\begingroup$

I found this nice-ish-looking diagram, but it has a wholly inadequate descriptions for each of the cell types, aside from including names.

What is the definition/description of each of these cell types?

  • Input Cell
  • Backfed Input Cell
  • Noisy Input Cell
  • Hidden Cell
  • Probablistic Hidden Cell
  • Spiking Hidden Cell
  • Output Cell
  • Match Input Output Cell
  • Recurrent Cell
  • Memory Cell
  • Different Memory Cell
  • Kernel
  • Convolution or Pool
$\endgroup$
  • $\begingroup$ Welcome to AI. I took the liberty of adding a couple of tags. Useful question! $\endgroup$ – DukeZhou Apr 4 '18 at 17:52
  • $\begingroup$ check this out ai.stackexchange.com/questions/5546/… $\endgroup$ – DuttaA Apr 4 '18 at 17:56
  • $\begingroup$ Yea, that is helpful, and definitely in the direction that I am hoping for an answer. $\endgroup$ – adv Apr 4 '18 at 19:17
  • $\begingroup$ This one gets closer: medium.com/@culurciello/… Though it is a bit lacking in comprehensive coverage. $\endgroup$ – adv Apr 6 '18 at 15:20
2
$\begingroup$

The diagram is nice-ish-looking, and it does lack a convenient list of node type descriptions. There is some description in their Zoo Flashcards page. Although the zoo is not a clean presentation in every respect, it is quite instructive in several ways. These are of particular interest.

  • The symbolic shapes and colors visually display the relationship between cell types and the types of networks that employ them, which is quite instructive as an overview.
  • The use of the term cell, as opposed to neuron, acknowledges the fundamental difference between current artificial network cells and the amazingly intricate and self-modifying nature of biological neurons.

What is still missing is a concise list of each cell type and its purpose for existing in the contexts of its use. This may be useful.

  • Input Cell — Provides a point of input and often uses an identity activation function (the output = the input)
  • Back-fed Input Cell — Provides a point where feedback is deliberately introduced
  • Noisy Input Cell — Provides a point where chaotic (pseudo-random), thermal, or quantum noise can be introduced
  • Hidden Cell — A cell that attenuates (applies a parameterized weight to) the input from all or a select range of activation function outputs from the cells of the immediately prior layer
  • Probabilistic Hidden Cell — Applies a radial basis function to the difference between the test case and the cell's mean1
  • Spiking Hidden Cell — Employs a spiking model of activation rather than a perceptron model, working toward a more accurate simulation of biological neurons at the cell level
  • Output Cell — Provides a point of output and applies the range (min and max) or ordinality (binary, category index)
  • Match Input Output Cell — Used in auto-encoders to converge on the identity function from network input to network output, usually through a narrowed data path through which features are extracted after convergence
  • Recurrent Cell — Used in RNN designs such as recurrent artificial networks as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
  • Memory Cell — Used in LSTM (long short term memory) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
  • Different Memory Cell — Used in GRU (gated recurrent unit) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
  • Convolution Cell — Passes the result of a convolution kernel multiplied to a relative range of the cells of the immediately previous layer to the next layer
  • Pool Cell — Aggregates (usually arithmatically) the results of the cells of a convolution layer

A table of cell types with the features for each might be a next step in presenting the cell types in a comparative summary. The table would particularly helpful if it had sort up and down gadgets in the header row for each column.

Footnotes

[1] Typically $\varphi(\mathbf{x}) = \sum_{i=1}^I a_i e^{\left[ -\beta \left \Vert \mathbf{x} - \mathbf{c}_i \right \Vert ^2 \right]}$ where c is the center value of the cell

[2] These cell types integrate historical influence into the signal path at the cellular level, introducing statefulness beyond the learning parameters and network hyper-parameters, providing the potential of Turing completeness to the network after training.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.