I know the basics of Artificial Neural Networks. For instance; make dot product with the weights and every neuron from previous layer. Adjust the weight by error. And done, That is how I see neural networks, but I saw in lot of videos in the graphical representation that every neuron has it's synapse with certain neurons. And when it change the input for example they create new synapses with other certain neurons. So whats what I'm asking for is:

How I can make my neurons assign the synapses with the neuron that fits it?

  • 1
    $\begingroup$ I believe a weight w = 0 means disconnected synapse $\endgroup$ – A.Rashad Aug 29 '17 at 19:52
  • $\begingroup$ I believe is something else...Why? Because weight very rarely is precisely 0. And a weight I think is neuron side, so if it is 0, it would render neuron useless in future use $\endgroup$ – Laceanu George Aug 29 '17 at 20:24
  • $\begingroup$ are you conflating between "neural networks in the brain" and "neural networks in AI for the past 10 years"? theory has inspired both sides, but they are very much different $\endgroup$ – k.c. sayz 'k.c sayz' Aug 29 '17 at 21:00
  • $\begingroup$ Would it be |w| << $epsilon$? $\endgroup$ – A.Rashad Aug 30 '17 at 12:16
  • 1
    $\begingroup$ |w| << $epsilon$ what does this mean? If it is something from phython or any other language, I'm so sorry but I have done programming only in c based languages $\endgroup$ – Laceanu George Aug 30 '17 at 13:41

What directs neuroplasticity to form memories and new behavior is definitely interesting.

There are a few inaccuracies in the question. Axons grow in directions and synapses form when they encounter another cell body and the cell grows dendrites to accept the connection. What determines the direction of growth and how the target cell decides to except or reject the connection is interesting too.

The description of multi-layer perceptron multiplication is not quite accurate. The parameter matrix is multiplied with the previous layer's output vector, but it is not a dot product. It's a vector-matrix multiplication, which is like a collection of dot products, one for each component of the vector product.

Simplifying back propagation and stochastic gradient descent as, "Adjust the weight by error and done," might insult those that have developed the science and art of selecting network cell types, activation functions, data preparation and selection, properly initializing parameters, setting hyper-parameters, and choosing methods of descending to work well with specific learning problems.

Here's where the question gets interesting and is perfectly accurate: "Every neuron has it's synapse with certain neurons." Either a synapse is functional, in the process of becoming so, or not at all. There is no universal pattern that's obvious, although organization is very apparent. There are structural changes within neurons too, in the cytoplasm and among the organelles.

The idea behind a multi-layer perceptron was that the connections between cells in adjacent layers that exist would have parameter values not equal to zero. Those that are zero are equivalent to a non-existent connection between neurons. The idea is sound but not particularly efficient, which is why there are designs with partially connected layers.

So the question, "How I can make my neurons assign the synapses with the neuron that fits it?" isn't as important as, "How can I decide which connections fit the purpose of my network?" The next question is, "Can I program the 'how I decide' of the previous question so that the network behaves neuroplastically and develops its own new structures that work in their role and work efficiently?"

Researchers have offered hypotheses based on association. An early idea was that the concurrency of pulse patterns is the basis for directing axon growth. If synchronization appears between two cells, they will tend to connect. That hypothesis is an older one, so that we don't hear much about it today probably means that not enough evidence supports it to draw a conclusion.

The more contemporary hypothesis for axon guidance involves attractive or repulsive forces. Some of the research is into molecular tags and guidepost cells.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.