# Tag Info

6

Really short answer: yes Slightly longer answer: kinda Long answer: Convolutional neural networks (CNNs), which are now a standard in image processing models, were inspired from work done by Hubel and Wiesel in the 1950-60s. They showed that the visual cortexes of cats and mokeys contain neurons which individually respond to small regions of the visual ...

5

It depends on the architecture of the neural network. However, in general, no, neurons at layer $l$ are not only affected by neurons at layer $l-1$. In the case of a multi-layer perceptron (or feed-forward neural network), only neurons at layer $l-1$ directly affect the neurons at layer $l$. However, neurons at layers $l-i$, for $i=2, \dots, l$, also ...

4

Only a small portion of the habituation, sensitization, and classical conditioning behavior of neurons has been primitively simulated in ANN systems. Simulation of actin cytoskeletal machinery1 and other agents of neural plasticity, central to learning new domains, is in its beginnings2. As of this writing, the complexity of neuron activation dwarfs the ...

4

It looks like you really have two questions here. I'll try to answer the first one, and you should think about making a separate question for the second. There is research into using simulated models of biologically realistic neurons. While there are large projects like the Human Brain Project aimed at simulating human brains, there is also a lot of lower-...

4

No, here is why. No approach can simulate the mind with 100% accuracy. a major notion that AI theorist refuse to note is that you cant take an orange and by virtue of technology turn it into an apple lets apply the same logic here. neurons are temporary things in our brains, daily we are trimming our brains and growing our brains, in order to "Engineer" a ...

4

State of Rosehip Research The Rosehip neuron is an important discovery, with vast implications to AI and its relationship to the dominant intelligence on earth for at least the last 50,000 years. The paper that has spawned other articles is Transcriptomic and morphophysiological evidence for a specialized human cortical GABAergic cell type, Buldog et. al.,...

4

The basic calculation for a single neuron is of the form $$\sigma\left(\sum_{i} x_i w_i \right),$$ where $x_i$ is the input to the neuron $w_i$ are the neuron-specific weights for every single input and $\sigma$ is the pre-specified activation function. In your terms, and disregarding the activation function, the calculation would turn out to be c\,a_c ...

4

In short I mentioned in another post, how the Artificial Neural Network (ANN) weights are a relatively crude abstraction of connections between neurons in the brain. Similarly, the random weight initialization step in ANNs is a simple procedure that abstracts the complexity of central nervous system development and synaptogenesis. A bit more detail (with ...

3

It depends on the accuracy you want. If you had 1 neuron, it could discern things across a line, if you have 2, you could solve things across 2 lines, etc. As you increase the number of neurons, you are increasing the number of discernible areas. As you increase the number of lines you can use to break up the input space, the lines can be placed to ...

3

Yes, this is actually a limitation known as catastrophic forgetting. A proposed way to deal with this is elastic weight consolidation that "remembers old tasks by selectively slowing down learning on the weights important for those tasks". See Overcoming catastrophic forgetting in neural networks for details. Another approach is Learning without forgetting. ...

3

It is true that the current Machine learning is based on treating neurons as a component in the whole complexity , mesh of neurons. The focus is more on the architecture rather than understanding or imitating the basic block of it more clearly , i.e. the neurons. Anirban Bandhopadhyay is a biologist and Neurologist who has studied how the harmony changes ...

3

Principles of Computational Modelling in Neuroscience by David Sterratt, Bruce Graham, Andrew Gillies and David Willshaw discuss it in Chapter 7 (The synapse) and also in Chapter 8 (Simplified models of neurons). Especially in chapter 8, they discuss how to add excitatory or inhibitory synapses to integrate and fire neuron. There are various ways to add ...

3

There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time. Many networks have been tried for text generation, with notable examples such as Markov chain. However RNN have been proven to work the best and is dominating the field of language modelling (...

3

In the case of artificial neural networks, your question can be (partially) answered by looking at the definition of the operation that an artificial neuron performs. An artificial neuron is usually defined as a linear combination of its inputs, followed by the application of a non-linear activation function (e.g. the hyperbolic tangent or ReLU). More ...

2

Sorry, this is a very broad area. Proper understanding of neural networks requires advanced mathematics. It's not sufficient to say "balancing of weights and biases" because most ML algorithms have weights. You seriously need to grab a book. OCR system itself is also very broad, it includes various object recognition techniques. You haven't even mentioned ...

2

ANNs approximate biological neuronal networks. The approximation began with extreme simplicity in the early perceptron design. Spiking networks are examples of more accurate approximations. More accurate still, are complex simulations of neuron behavior that therefore necessitate significant computing resources. If you are interested in a mathematical ...

2

I don't know if my intuition is correct but I will give it a try. You could see weights as how much important one thing is, the problem is to understand what that thing represents. When I say thing I'm referring to the output of a specific neuron. I don't think that we can say what the output of a neuron represents in the real world unless we directly ...

2

From your question I can assume you are a beginner in the field of AI. Welcome to this exciting field. To answer your question, we have not yet been able to create a truly artificially intelligent program. They are all apparently intelligent but are just a set of simple/complex rules. An artificially intelligent agent must have at-least 2 aspects in the ...

2

In biology, when the presynaptic releases a neurotransmitter (a positive amount of them, obviously), this neurotransmitter reaches the postsynaptic vesicles causing an excitatory (depolarization) or inhibitory (hyperpolarization) effect, depending on the kind of postsynaptic vesicle in next cell dendrites. If the total amount of depolarization (all dendrites)...

2

Recurrent Neural Networks (RNNs) have been applied to generate text. In this blog post you will find a couple of interesting text examples (the author also has made his code available on github), e.g. their Shakespeare-like texts generated by an RNN: PANDARUS: Alas, I think he shall be come approached and the day When little srain would be attain'd ...

2

Simon Krannig's answer provides the math notation behined exactly what is going on, but since you still seem a bit confused, I've made a visual representation of a neural network using only weights with no activation function. See below: So I'm fairly sure it as you suspected: At each neuron, you take the sum of the inputs of the previous layer multiplied ...

2

Yes, for many sensory inputs there is indeed something similar to normalization. But its not rally the same as in classical data analytics compared to what eg min/max normalization does or other technics. Lets look on some examples and considerations: mammals don't perceive heat or loudness in a linear way. This is because already many sensory receptors ...

2

In terms of the normal use cases for machine learning, the equation does not have much utility, because: Consider we have a curve $f(x)$ now if one wishes to . . . In most AI problems, we don't usually have such a curve as input that can be treated analytically. For instance, there is no such input curve to describe a natural image received by a sensor....

2

The cell of a perceptron was based on an oversimplified conception of a neuron. At the time, neural plasticity, timing factors in relation to activation, neurochemical pathways, and energy transit complexities in axons were unknown. The mapping of pulse transmission to basic algebra seemed unrealistic, so timing was ignored. Plasticity, timing, and regional ...

2

This should be possible given the fact that ANNs have the ability to do the feature engineering and feature selection tasks by themselves. This means that given a lesser number of input parameters, the model will be able to generate and select additional features by itself. You will obviously not be able to understand or model these features manually. The ...

2

Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer? I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the ...

2

To answer your question, I'll just simplify it and assume you are representing the activation of a neuron (the value it produces) by an RGB value. So a tuple with 3 values ranging from 0 to 1. It's important to remember that every single machine learning model could be computed by a human on pen and paper using just numbers. So keeping that in mind, this ...

1

Yes. Text summarisation has been a research topic in (computational) linguistics for literally decades. Have a look at the Wikipedia page on Automatic Summarisation for an overview. There are basically various different approaches: either, selecting salient sentences (or parts of sentences) which represent the gist of the text, or, trying to 'understand' ...

1

I see no problem in regarding the softmax as a particular activation function which takes a vector input and produces a vector output. In fact, the sigmoid function can be viewed as a two-dimensional softmax in which one of the two inputs is hardwired to zero while the corresponding output is neglected.

1

If the number of input neurons and output neurons doesn't change, what will change if I have one hidden layer, but first with 1 neuron, then with 4 neurons? I recommend that you try to implement a dense layer with Tensorflow (not using the Dense implementation, of course). A dense layer is the full connection between to layers. What actually is learned/...

Only top voted, non community-wiki answers of a minimum length are eligible