Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
7
votes
2answers
683 views

How to deal with large (or NaN) neural network's weights?

My weights go from being between 0 and 1 at initialization to exploding into the tens of thousands in the next iteration. In the 3rd iteration, they become so large that only arrays of nan values are ...
7
votes
4answers
891 views

Is it suitable to find inverse of last layer's activation function and apply it on the target output?

I have a neural network with the following structure: I am expecting specific outputs from the neural network which are the target values for my training. Let's say the target values are 0.8 for the ...
7
votes
4answers
7k views

Traffic signs dataset [closed]

I'm looking for annotated dataset of traffic signs. I was able to find Belgium, German and many more traffic signs datasets. The only problem is these datasets contain only cropped images, like this: ...
7
votes
3answers
557 views

How to model inhibitory synapses in the artificial neuron?

In the brain, some synapses are stimulating and some inhibiting. In the case of artificial neural networks, ReLU erases that property, since in the brain inhibition doesn't correspond to a 0 output, ...
7
votes
3answers
866 views

How do I know if my backpropagation is implemented correctly?

I'm working on implementation of the backpropagation algorithm for a simple neural network which predicts a probability of survival (1 or 0) and I can't get it above 80% no matter how much I try to ...
7
votes
2answers
7k views

Which neural network has capabilities of sorting input?

I believe normally you can use genetic programming for sorting, however I'd like to check whether it's possible using ANN. Given the unsorted text data from input, which neural network is suitable ...
7
votes
1answer
87 views

Medical diagnosis systems based on artificial neural networks

Are there any medical diagnosis systems that are already used somewhere that are based on artificial neural networks?
7
votes
1answer
86 views

What makes multi-layer neural networks able to perform nonlinear operations?

As I know, a single layer neural network can only do linear operations, but multilayered ones can. Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can ...
7
votes
1answer
2k views

What do prediction/mean and label/mean represent in this TensorFlow code? [closed]

I am pretty much a beginner in Tensorflow and simply follow a tutorial. There is no problem with my code, but I have a question regarding the output ...
7
votes
1answer
3k views

What is the definition of each of these neural network cell types?

I found this nice-ish-looking diagram, but it has a wholly inadequate descriptions for each of the cell types, aside from including names. What is the definition/description of each of these cell ...
7
votes
1answer
115 views

Which challenges in information security can be solved better using neural networks?

Information security has become a thriving field during the last years. It is a broad domain ranging from planing and building over testing to operating different applications, systems and networks in ...
7
votes
2answers
2k views

How to write a C decompiler using AI?

I would like to learn more about whether it is possible and how to write a program that decompiles executable binary (an object file) to the C source. I'm not asking exactly 'how', but rather how this ...
7
votes
3answers
203 views

To what does the number of hidden layers in a neural network correspond?

In a neural network, the number of neurons in the hidden layer corresponds to the complexity of the model generated to map the inputs to output(s). More neurons creates a more complex function (and ...
7
votes
1answer
685 views

How should we choose the dimensions of the encoding layer in auto-encoders?

How should we choose the dimensions of the encoding layer in auto-encoders?
7
votes
1answer
484 views

How can a neural network approximate all functions when the weights are not allowed to grow exponentially?

It has been proven in the paper "Approximation by Superpositions of a Sigmoidal Function" (by Cybenko, in 1989) that neural networks are universal function approximators. I have a related question. ...
7
votes
3answers
479 views

CNN's vs Densely Connected NN's

In image classification we are generally told the main reason of using CNN's is that densely connected NN's cannot handle so many parameters (10 ^ 6 for a 1000 * 1000 image). My question is, is there ...
7
votes
1answer
643 views

Why aren't there neural networks that connect the output of each layer to all next layers?

Why aren't there neural networks that connect the output of each layer to all next layers? For example, the output of layer 1 would be fed to the input of layers 2, 3, 4, etc. Beyond computational ...
7
votes
2answers
3k views

Does data skew matter in classification problem?

I'm working on a image classification problem using neural-network. In the training data set, 90% of the samples fall into 10% of all categories, while 10% of the sample fall into the other 90% ...
7
votes
1answer
737 views

What is the intuition behind the attention mechanism?

Attention idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to "look back” at the complete input and extracts ...
7
votes
1answer
279 views

Are there existing examples of using neural networks for static code analysis?

Background Context: In the past I've heavily applied various "code quality metrics" to statically analyze code to provide an inkling of how "maintainable" it is and using things like the ...
7
votes
1answer
7k views

How is the gradient calculated for the middle layer's weights?

I am trying to understand backpropagation. I used a simple neural network with one input $x$, one hidden layer $h$ and one output layer $y$, with weight $w_1$ connecting $x$ to $h$, and $w_2$ ...
7
votes
1answer
410 views

What is the relationship between these two taxonomies for machine learning with neural networks?

Could you please let me know which of the following classification of Neural Network's learning algorithm is correct? The first one classifies it into: supervised, unsupervised and reinforcement ...
7
votes
1answer
818 views

How come that the addition of features can decrease the performance of a neural network?

I have a Remaining Useful Life (RUL) prediction problem that I want to solve. When I added two or more features as inputs to my ANN, the accuracy of my ANN has been decreased. More precisely, I've ...
7
votes
1answer
2k views

Fuzzy Logic Controller: Choosing Membership Function

In classical set theory there is two options for an element. It is either a member of a set, or not. But in fuzzy set theory there are membership functions to define "rate" of an element being a ...
7
votes
2answers
597 views

Is it possible to implement reinforcement learning using a neural network?

I've implemented the reinforcement learning algorithm for an agent to play snappy bird (a shameless cheap ripoff of flappy bird) utilizing a q-table for storing the history for future lookups. It ...
7
votes
2answers
423 views

What are the best machine learning models for music composition?

What are the best machine learning models that have been used to compose music? Are there some good research papers (or books) on this topic out there? I would say, if I use a neural network, I would ...
7
votes
2answers
145 views

Can we teach an artificial intelligence through sentences?

Could we teach an AI with sentences such as "ants are small" and "the sky is blue"? Is there any research work that attempts to do this?
7
votes
1answer
1k views

Loss functions for Hierarchical Multi-label classification?

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
6
votes
9answers
1k views

Can Neural Networks self-optimize?

Suppose that you show a neural network its own code, and allow it to edit itself? Can a neural network modify its own weights and architecture (the number of layers, the number of neurons per layer, ...
6
votes
6answers
2k views

How to evaluate a NEAT neural network?

I'm trying to write my own implementation of NEAT and I'm stuck on the network evaluate function, which calculates the output of the network. NEAT as you may know contains a group of neural networks ...
6
votes
3answers
2k views

Convolutional Neural Network: does each filter in each convolution layer create a new image?

Say I have a CNN with this structure: input = 1 image (say, 30x30 RGB pixels) first convolution layer = 10 5x5 convolution filters second convolution layer = 5 3x3 convolution filters one dense layer ...
6
votes
5answers
327 views

Emulating human brain - with analogous NN chips

Considering the answers of this question, emulating a human brain with the current computing capacity is currently impossible, but we aren't very far from it. Note, 1 or 2 decades ago, similar ...
6
votes
3answers
307 views

What kinds of problems can AI solve without using a deep neural network?

A lot of questions on this site seem to be asking "can I use X to solve Y?", where X is usually a deep neural network, and Y is often something already addressed by other areas of AI that ...
6
votes
2answers
4k views

Can a neural network with linear activation functions produce a connection of linear functions?

From a lecture in machine learning, I know that a linear activation function can only produce a linear function, but I don't know if it can produce a connected linear function, like the one in the ...
6
votes
4answers
3k views

What is the purpose of the hidden layers?

Why would anybody want to use "hidden layers"? How do they enhance the learning ability of the network in comparison to the network which doesn't have them (linear models)?
6
votes
1answer
357 views

How to express a fully connected neural network succintly using linear algebra?

I'm currently reading the paper Federated Learning with Matched Averaging (2020), where the authors claim: A basic fully connected (FC) NN can be formulated as: $\hat{y} = \sigma(xW_1)W_2$ [...] ...
6
votes
3answers
237 views

Is there any artificially intelligent system that really mimics human intelligence?

After having read something that Elon Musk said about artificial intelligence and how it could affect our lives, I've been reading about artificial intelligence, deep learning, etc. The recurrent ...
6
votes
1answer
4k views

How to avoid falling into the "local minima" trap?

How do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network? Are there any methods which help me avoid it?
6
votes
2answers
292 views

Why is dropout favoured compared to reducing the number of units in hidden layers?

Why is dropout favored compared to reducing the number of units in hidden layers for the convolutional networks? If a large set of units leads to overfitting and dropping out "averages" the response ...
6
votes
1answer
1k views

Is this idea to calculate the required number of hidden neurons for a single hidden layer neural network correct?

I have an idea to find the optimal number of hidden neurons required in a neural network, but I'm not sure how accurate it is. Assuming that it has only 1 hidden layer, it is a classification problem ...
6
votes
1answer
523 views

Can a vanilla neural network theoretically achieve the same performance as CNN?

I perfectly understand that CNN takes into account the local dependency of each pixel to the nearby pixels. In addition, CNNs are spatially invariant which means that they are able to detect the same ...
6
votes
2answers
1k views

How is local minima possible in gradient descent?

Gradient descent works on the equation of mean squared error, which is an equation of a parabola $y=x^2$. We often say that weight adjustment in a neural network by gradient descent algorithm can hit ...
6
votes
1answer
141 views

Can this tic tac toe program be considered AI?

I coded a tic tac toe program, but I don't know if I can call it artificial intelligence. Here's what I did. There is a random player, which always makes random valid moves. And then there is the ...
6
votes
1answer
400 views

Why not teach to a NN not only what is true, but also what is not true?

I'm not a person who studies neural networks, or does anything that is related with that area, but I have seen a couple of seminars, videos (such as 3Blue1Brown's Series), and what I am always told is ...
6
votes
1answer
571 views

What if the more fit parent has fewer nodes compared to the other, will the disjoint and excess genes be discarded?

In the paper Efficient Evolution of Neural Network Topologies (2002), the authors say Genes that do not match are inherited from the more fit parent What if the more fit parent has fewer nodes ...
6
votes
1answer
4k views

Why most imperfect information games usually use non machine learning AI?

To provide a bit of context, I'm a software engineer & game enthusiast (card games, especially). The thing is I've always been interested in AI oriented to games. In college, I programmed my own ...
6
votes
2answers
194 views

How to shorten the development time of a neural network?

I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After ...
6
votes
1answer
441 views

How do neural networks manage to do regression?

I'm trying to learn about neural networks, and I'm interested in gaining a better conceptual understanding of how they work to solve certain problems. I'm having trouble in conceptually understanding ...
6
votes
4answers
4k views

What is the purpose of hidden nodes in neural network?

If I have a set of sensory nodes taking in information and a set of "action nodes" which determine the behavior of my robot, why do I need hidden nodes between them when I can let all sensory nodes ...
6
votes
4answers
808 views

What are some datasets to train an MLP on simple tasks? [closed]

I have implemented an MLP. Now, I want to train it to solve simple tasks. Are there any data sets to train the MLP on simple tasks, that is, tasks with a small number of inputs and outputs? I ...

1 2 3
4
5
45