10
$\begingroup$

A neural network is a directed weighted graph. These can be represented by a (sparse) matrix. Doing so can expose some elegant properties of the network.

Is this technique beneficial for examining neural networks?

$\endgroup$
2
  • $\begingroup$ This is for Cross Validated SE. $\endgroup$ Aug 5 '16 at 18:35
  • $\begingroup$ Hi. Was any of the below answers helpful? If yes, the please consider accepting one :) $\endgroup$
    – Dawny33
    Feb 16 '17 at 19:44
4
$\begingroup$

For large ANNs, something equivalent to a 'sparse matrix format' is used in practice.

In contrast to what is said in another answer given, considering an ANN as a graph doesn't actually buy very much, for two reasons:

  1. The backpropagation algorithm can usefully be defined in terms of matrix operations. This page gives a readable and comprehensive description.

  2. All real-valued matrices can be represented as graphs, but the converse is clearly not the case. So while it is true that an ANN can be considered as a special case of a graph data structure, making that specialization explicit in matrix form is more efficient.

$\endgroup$
4
$\begingroup$

It depends on the type of neural networks you are dealing with.

For medium sized neural nets, the matrix approach is a very good way to do quick computations and even backpropogation of errors. One can even exploit sparse matrixes for understanding the sparse architecture of some neural nets.

But, for very large neural nets, using matrix computations would be computationally very intensive. So, relevant methods like graph-based stores, etc are used for them depending on the purpose and the architecture.

$\endgroup$
1
$\begingroup$

Matrix representation is beneficial for implementing neural networks in silicon.

But for examining neural networks empirically it is sometimes good to visualise the synapse weight values as images or videos: Jason Yosinski's exploration of a convolution neural network. The network seems to have a "filter" that just detects shoulders. A bit like a lock that only opens when it recognises the pattern of shoulders.enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.