I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip.

If, instead of layers that would be fully connected, I have layers with neurons connected in some other way (some pairs of neurons connected, others not), how is this going to affect the hardware acceleration?

An example of this is the convolutional networks. However, there is still a clear pattern, which perhaps is exploited by the acceleration, which would mean that if there is no such pattern, the acceleration would not work as well?

Should this be a concern? If so, is there some rule of thumb for how the connectivity pattern is going to affect the efficiency of hardware acceleration?

  • 3
    $\begingroup$ The answer is going to depend on details - e.g. how sparse the connections are, whether they can be arranged meaningfully into layers etc. Also whether you have training data available in large batches. Do you have any specific use case or scenario that would narrow the scope down a little? Or are you looking for a broad but shallow answer? $\endgroup$ Sep 23 '19 at 19:00
  • $\begingroup$ I am interested in a broad but shallow answer. Or even better, to some place where I could read more. $\endgroup$ Oct 3 '19 at 18:02
  • $\begingroup$ Yes, it matters, and I think you have to manually decide how to divide it up, and hello pre-covid-19 question. $\endgroup$
    – user253751
    Sep 2 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.