Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

0
votes
0answers
8 views

Understanding Hebb network

Can anyone help me in understanding Hebb networking and how different function like AND, OR used to solve by this network. I couldn’t understand properly through the google.
0
votes
0answers
9 views

How to properly optimize shared network between actor and critic?

I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment. When I share the encoder with the actor ...
1
vote
0answers
11 views

Point Cloud Alignment using a Neural Network?

Having two point clouds, the second being a transformation of the first, how could I utilize a neural network in order to solve the pose (transformation in terms of x, y, z, rx, ry, rz) of the second ...
0
votes
1answer
21 views

Alphazero policy head loss not decreasing

I am now working on training an alphazero player for a board game. The implementation of board game is mine, MCTS for alphazero was taken elsewhere. Due to complexity of the game, it takes a much ...
2
votes
1answer
27 views

Does reinforcing correct predictions increase model accuracy further?

Let's say I've trained a CNN that is predicting/inferring live samples that it hasn't seen before. In the event the network makes a correct prediction, would including this as a new sample in its ...
1
vote
0answers
23 views

What are some examples of tasks in which, currently, neuroevolution outperforms gradient-based approaches?

Note: I am NOT asking for general advantages of neuroevolution over standard approaches (e.g.: architecture search, parallelization), I am asking for examples of tasks in which, currently, ...
5
votes
2answers
52 views

How is local minima possible in gradient descent?

Gradient descent works on the equation of mean squared error, which is an equation of a parabola $y=x^2$. We often say that weight adjustment in a neural network by gradient descent algorithm can hit ...
1
vote
1answer
61 views

What is an artificial neural network?

What is an artificial neural network in artificial intelligence? It is apparently used to find patterns in data and it is loosely inspired by human neural networks.
0
votes
1answer
22 views

Binary vector expected value

Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1 calculates the variance of the output of a hidden neuron. Raul Rojas says that "for binary vectors we have $E[x_i^2] = \frac{1}{3}$...
0
votes
0answers
18 views

Why the loss is nan by using linear activation function in the last layer? [migrated]

I want to use neural network to solve a simple regression problem, and I try to program by myself accroding to lecture Backpropagation and Neural Networks . However, I meet loss divergence problem. ...
1
vote
1answer
25 views

Does changing the order of the convolution layers in a CNN have any impact?

Could changing the order of convolution layers in a CNN improve accuracy or training time?
0
votes
0answers
23 views

What is the relation between neural embedding and neural code?

Lets consider knowledge graph and operations on it. There are notions of neural embedding and neural coding for it. What is the relation between neural embedding and neural code? Is neural coding a ...
1
vote
1answer
22 views

AlphaGo neural network inputs

I have two questions: 1) I have been reading an article on AlphaGo and one sentence confused me a little bit, because I'm not sure what it exactly means. The article says: AlphaGo Zero only uses ...
3
votes
1answer
22 views

How should I encode a categorical input?

Let's say you have an input which can take one of 10 different unique values. How would you encode it? Have input length 10 and one-hot encode it. Have 1 input but normalise the value between the ...
1
vote
0answers
31 views

How are the weights between the input and hidden layer updated in a 3 layer neural network?

Consider a feed-forward neural network with one hidden layer. How are the weights between the input and hidden layer updated, after the weights between the hidden layer and output layer are updated?
3
votes
1answer
35 views

Variational Autoencoder task for better feature extraction

I have a CNN with the regression task of a single scalar. I was wondering if an additional task of reconstructing the image (used for learning visual concepts), seen in a DeepMind presentation with ...
3
votes
1answer
52 views

Why is dot product attention faster than additive attention?

In section 3.2.1 of Attention Is All You Need the claim is made that: Dot-product attention is identical to our algorithm, except for the scaling factor of $\frac{1}{\sqrt{d_k}}$. Additive ...
3
votes
0answers
44 views

What are stable ways of doing online machine learning?

I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-...
2
votes
1answer
48 views

How do I choose the number of neurons in the fully-connected layer before the softmax layer?

I am solving a classification problem with CNN. The number of classes is 5. How can I decide the number of neurons in the FC layer before the softmax layer? Is it $N * 5$, where $N$ is the number of ...
0
votes
0answers
55 views

neural net editing neural net and evolution [on hold]

First of all, there is a neural network. Interaction between neural networks (input/output exchange, editing). These (neural networks) interact like life. The beginning is evolutionary algorithm. ...
4
votes
1answer
205 views

What loss function to use when labels are probabilities?

What loss function is most appropriate when training a model with target values that are probabilities? For example, I have a 3-output model. I want to train it with a feature vector $x=[x_1, x_2, \...
-1
votes
0answers
58 views

Can differential equations be replaced by a neural networks?

Introduction Artificial Intelligence can be realized in many ways. A common criteria is to distinct between Narrow AI and Strong AI. Strong AI is often described as a cognitive architecture which will ...
1
vote
1answer
21 views

Machine learning to find drivers of an event with presence-only data (no absence)

I have some ecological data on the confirmed presence of a certain animal. I have data on the: ...
0
votes
0answers
27 views

Why does precision-recall curve become more stable when neural net begins to overfit?

I am training a convLSTM with a dropout layer (with prob 0.5). If I train over more than 5 epochs I notice that the network starts to overfit: my validation set loss becomes stationary while the ...
1
vote
1answer
45 views

Can I use neural networks for a problem (in description)?

I am modelling a process with 4 input parameters x1 x2 x3 x4. The output of the process is 2 variable y1 y2that varies with ...
0
votes
0answers
30 views

Actor-critic algorithm using gaussian Radial Basis Function, Local Linear Regression and shallow Neural Network

I'm attempting to implement the actor-critic algorithm on Matlab using Radial Basis Function, Local Linear Regression, and shallow Neural Network for inverted pendulum system. the state space and the ...
4
votes
1answer
61 views

Is it possible to make a 'forked path' neural network?

I want to make a network, specifically a CNN for image recognition, that takes an input, processes it the same way for several layers, and then at some point splits before coming to two different ...
1
vote
0answers
33 views

What are the key differences between cellular neural network and convolutional neural network?

What are the key differences between cellular neural networks and convolutional neural networks in terms of working principle, implementation, potential performance, and applicability?
3
votes
1answer
40 views

Why are not validation accuracy and loss as smooth as train accuracy and loss?

I am training a modified VGG16 network for classification (adding 0.5 dropout after each of the last FC layers). In the following plot I am training for a small number of epochs as an example, and it ...
2
votes
0answers
17 views

How do we ensure that training GANs will fall in the desirable Nash equilibrium?

One Nash equilibrium of every GANs model has is when the generator creates perfect samples indistinguishable from the training data and the discriminator just output 1 with probability 1/2. And I ...
2
votes
0answers
12 views

Models of reward (possibly mimicking dopamine) in artificial neural networks?

How one can model physiological reward mechanisms occuring in the brain using artificial neural networks? E.g. are there efforts to use the notion of dopamine or similar substances in the artificial ...
1
vote
0answers
13 views

How to predict a preferred route based on weather and distance

I want to train a neural network to predict what my favourite home-work route will be for a particular day. I have these features for routes on a day: ...
0
votes
0answers
12 views

Using the cloud service to trasform a picture using a neural algorithm?

yesterday I tried to transform a picture in the artistic style using CNNs based on A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge using a recent Torch ...
2
votes
1answer
19 views

Why do DQNs use linear activations on cartpole?

I've been reading a lot of tutorials on DQNs for cartpole. In many of them, they have the funnel layer of the neural net be a linear activation. Why is this? Is it just a choice made by the ...
2
votes
0answers
32 views

Algorithms to indentify people in pictures without using face recognition

There are lot of researches about face detection in pictures, but is it the only way one can say "this person I'm looking for is here in this picture"? Aren't there algorithms that you can provide ...
2
votes
0answers
35 views

RNN: Different test results on balanced and unbalanced data

I trained a recurrent neural network (if it matters - it contains three CuDNNLSTM cells and 3 Dense layers, Dropout = 0.2). The result of data preparation is one array of ~330.000 sequences. Each ...
2
votes
1answer
32 views

Can a neuron have both a bias and a threshold?

I have not seen a neuron that uses both a bias and a threshold. Why is this?
2
votes
1answer
24 views

Extending a neural network to classify new objects

Suppose a model M classifies apples and oranges. Can M be extended to classify a third class of objects, e.g., pears, such that ...
1
vote
1answer
37 views

What kind of functions can be used as activation functions?

I read that functions are used as activation functions only when they are differentiable. What about the unit step activation function? So, is there any other reason a function can be used as an ...
1
vote
1answer
37 views

Is back propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back propagation. I have a doubt on when the back propagation is applied. Assume that I have a training data set of 1000 images for ...
2
votes
2answers
40 views

Is max pooling really bad?

That has been discussion on this. Maybe from Hinton himself. And I heard that many max pooling layers have been replaced by conv layers in recent years, is that true?
0
votes
1answer
15 views

Back propagation on Flatten Layer in CNN

I am making a NN library without any other external NN lib and is implementing the Flatten layer. I know the forward implementation of flatten layer but is the backward just reshaping it or not? If ...
2
votes
0answers
42 views

How to build a neural network that can learn to predict output images?

I am working with a dataset where each input sample is a matrix, and the output corresponding to each input is also a matrix (of shape (400, 10)). The input samples ...
0
votes
0answers
27 views

Which neural network to use for mapping a vector of size m to a vector of size n, where n >> m?

I am trying to solve a mapping problem on a grid (100x100) where I have few points, say 10, where I know the values of a tensor $\boldsymbol{M}$. I have a scalar field, $v$, which is related to the ...
1
vote
1answer
52 views

Does backpropagation update weights one layer at a time?

I am new to Deep Learning. Suppose that we have a neural network with one input layer, one output layer, and one hidden layer. Let's refer to the weights from input to hidden as $W$ and the weights ...
3
votes
0answers
18 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
4
votes
1answer
90 views

What is a simplified way to explain why the AI researchers Bengio, Hinton, and Lecun, won the 2019 Turing Award?

The Turing award is sometimes called Computer Sceince's Nobel Prize. This year's award goes to Bengio, Hinton, and LeCun for their work on artificial neural networks. The actual work contributed by ...
1
vote
0answers
25 views

What methods are there to generate artificial training examples based on existing training examples?

I have a small dataset (117 training examples) and many features (4005). Each of the training examples is binary labeled (healthy / diseased). Each feature represents the connectivity between two ...
4
votes
1answer
60 views

Can this tic tac toe program be considered AI?

I coded a tic tac toe program, but I don't know if I can call it artificial intelligence. Here's what I did. There is a random player, which always makes random valid moves. And then there is the ...
1
vote
1answer
47 views

What are neural networks and how do they relate to AI?

Artificial neural networks (ANN) are computing systems vaguely inspired by the biological neural networks that constitute animal brains, how do they relate to AI?