4
$\begingroup$

I am not a researcher, but I am curious to know what considerations are relevant to take into account during research for the invention of a new neural network model, and what relevant knowledge researchers typically possess in the area.

And an accompanying question: is a background in neuroscience relevant to such an investigation?

Thank you very much for your time and guidance.

$\endgroup$
3
$\begingroup$

In my experience researchers typically base their architectures on previously identified successful architectures and principles. That is to say published methods that have been successful in practice on similar tasks to the current one. This can be followed back to to very early networks like the Perceptron which took a lot of inspiration from existing successful mathematical techniques.

This is not to say that researchers do not draw from neuroscience or from broader knowledge to influence their decisions. I believe most research is guided by more abstract intuitions derived from broader experience. Some researchers are very strongly influenced by neuroscience and I understand that some try to replicate behaviours seen in the brain. The similar structure and functions of the brain will surely make knowledge of it useful. Some major advances may have been motivated by direct analogy to the brain, although I am not aware if that is the case. But, on balance, having zero knowledge of neuroscience is not an impediment to undertaking successful research into neural architectures. Having zero knowledge of existing successful techniques is a far greater impediment.

Having a good level of mathematical understanding is also very useful. I will take perhaps a slight risk of going beyond my knowledge and suggest that university-level knowledge of mathematics is also more beneficial than knowledge of neuroscience. I think that is true for research so far, maybe it will change in future. All these comments are based only on my personal experience and not on any formal research. Also note my bias: I have minimal knowledge of neuroscience myself.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.