Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Mathematically speaking, what does the target networks in DDPG compute?

In general, to implement DDPG, we use four networks instead of two. They are actor, critic, target actor, and target critic. I am writing the mathematical formulation of the first two networks I ...
user avatar
  • 3,059
0 votes
0 answers
8 views

How to model a set-to-set mapping with graph neural networks?

I have a task on a heterogeneous graph where a set of nodes is given as input and some of the nodes are acceptable outputs. The dataset essentially consists of pairs (X, Y) where X is a set of nodes ...
user avatar
  • 101
0 votes
1 answer
29 views

How should you reshape data before feeding it to LSTM layers?

I was curious if anyone had any advice on how to reshape data for a recurrent neural network. What I've been doing is array.reshape(len(X_train), # of points in time, # of features) And then in the ...
user avatar
-1 votes
0 answers
17 views

Do all input nodes need to have a connection in the initial population [closed]

I know that in the initial population, all networks only have connections and no hidden nodes, but how many connections should be made?
user avatar
  • 1
0 votes
0 answers
22 views

Predicting using time-series data and static data?

I have recently been working on predicting the final value of articles on Steemit.com using downloaded data. I have a large variety of features which divide into two types. Features which change over ...
user avatar
0 votes
0 answers
14 views

What happens in tensorflow/pytorch under the hood when doing a convolution2d() with x filters when num input feature maps/channels > x? [duplicate]

I was wondering about what happens in tensorflow/pytorch under the hood when doing a convolution2d() with x filters when num input feature maps/channels > x? e.g. my input shape with 129 feature ...
user avatar
0 votes
0 answers
14 views

If two functions are close apart can I proof the difference of their empirical loss is also small?

I am trying to understand the proof of Theorem 3 in the paper A Universal Law of Robustness via isoperimetry by Bubeck and Sellke. Basically, there exist at least one $w_{L,e}$ in $\mathcal{W}_{L,e}$ ...
user avatar
0 votes
1 answer
18 views

Why gradients are used in Layer-wise Relevance Propagation (LRP)?

To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are ...
user avatar
  • 111
2 votes
1 answer
40 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
user avatar
1 vote
1 answer
45 views

Can you make a Neural Network drunk or high?

We know that the human brain can become sozzled by various substances that are released into the brain, but can you make an artificial neural network drunk or high? For example on a RL Agent that ...
user avatar
0 votes
0 answers
7 views

Is it possible to add identities into existed face recognition system without re-train the NN?

I have followed some tutorials and I find out all of them could not add new faces. If a new face is added, the system would have to be retrained. This doesn't sound logical because it would waste too ...
user avatar
  • 1
3 votes
1 answer
186 views

How does learning the moves of chess show up in a neural network?

Is learning the moves a special case or just the same sort of thing that happens as the AI learns strategy? If you take two different neural networks and teach them each how the pieces move, what ...
user avatar
  • 141
0 votes
0 answers
20 views

Encoding Actions with Parameters in Neural Network Output

I have a task which I would like to teach an AI to perform. The input to the task will a screenshot of the screen and the output at any given time step is one of the following actions: ...
user avatar
1 vote
0 answers
25 views

What do "large variables" and "small weights" mean in these sentences?

I'm trying to understand these two points from an article: Models with large variables i.e weight matrices. As a consequence such models have correspondingly large gradients and optimizer states. The ...
user avatar
  • 111
1 vote
1 answer
43 views

Is there some kind of "weighted maximum" that allows the gradients to backpropagate? [closed]

I was wanting to add a maximum in my neural network, but this seems a bad thing to do since it kills the gradients to all but one of the inputs. Is there some kind of "weighted maximum" that ...
user avatar
0 votes
0 answers
15 views

Recursive Memory Optimized Gradient Graph Explained?

I'm reading the paper Training Deep Nets with Sublinear Memory Cost by Tianqi Chen, et. al. The paper is known for the $O(\sqrt n)$ memory cost to train a $n$-layer neural network. My problem is ...
user avatar
  • 111
8 votes
1 answer
1k views

What do symmetric weights mean and how does it make backpropagation biologically implausible?

I was reading a paper on alternatives to backpropagation as a learning algorithm in neural networks. In this paper, the author talks about the disadvantages of backpropagation, and one of the ...
user avatar
  • 83
4 votes
1 answer
59 views

Why is it important/beneficial for an activation function to be zero-meaned?

Conventionally, (although there are plenty of better options) it is being said that as the choice of activation function for hidden layers, tanh should be prefered ...
user avatar
-1 votes
1 answer
30 views

How Many Hidden Units in an LSTM? [duplicate]

Is there any rule of thumb for choosing the number of hidden units in an LSTM? Is it similar to hidden neurons in a regular feedforward neural network? I'm getting better results with my LSTM when I ...
user avatar
  • 11
0 votes
2 answers
28 views

Should the range of target values match the range of activation function used in the output layer?

Suppose I use a tansig activation function in the output layer of an artificial neural network giving me outputs in the range $[-1,1]$ and my model is applied to a binary classification problem, ...
user avatar
-1 votes
1 answer
24 views

Is this the right approach to preprocessing data for artificial neural-networks? [closed]

I recently participated in a competitive "hackathon" with the problem being binary classification of overall satisfaction for travelers. The dataset mostly consisted of survey questions and ...
user avatar
0 votes
0 answers
10 views

Sketch-based segmentation attempt with Deep Learning

I'm looking for a deep-learning based segmentation capability, which should primarily consist of two steps: The image to be examined contains certain structures, which are mainly defined by geometric ...
user avatar
  • 131
1 vote
1 answer
35 views

What architecture would be best to match images of torn pieces of tapes?

I am currently working on a project where the goal is to create a neural network that can determine if two pieces of torn tapes are a true fit or not. My current idea is a convolutional network that ...
user avatar
0 votes
0 answers
33 views

What are some solid metrics to evaluate/compare the outputs of explainable algorithms?

Consider a learned CNN image classifier and a task that focuses on studying the outputs of explainable algorithms, such as integrated gradients and grad-cam, on the classifier's predictions. I am ...
user avatar
  • 113
0 votes
0 answers
15 views

When I give an input to a NN, it is generally assumed that the inputs are independent?

I am trying to find a pattern in various measures of health of an individual and using NN. I am using various parameters: Blood Work Some measure of heart condition - turning ECG reading into a ...
user avatar
0 votes
0 answers
16 views

Derivation in paper Deep Neural Networks as Gaussian processes in ICLR 2018

I am trying to understand the derivation of the main equation in the seminal paper titled Deep Neural Networks as Gaussian processes (in ICLR 2018). I have asked this question in https://math....
user avatar
0 votes
3 answers
78 views

How to deal with an unbalanced dataset?

I'm constructing a feed forward neural network that predicts whether a patient will get a stroke or not. However, my dataset is very unbalanced. Out of 5111 rows, 250 contain patients that have had a ...
user avatar
0 votes
0 answers
16 views

How does spatial pyramid pooling really work?

I went over the SPP paper (by Kaiming He) and understood in general that SPP can solve the fixed input size of the network problem (mainly due to the FC layer and not the CNN itself). Furthermore, the ...
user avatar
  • 1
0 votes
1 answer
27 views

How is catastrophic cancellation dealt with in loss functions?

It just occurred to me that this seems like it should be a very common problem that must have some kind of solution... Yet I'm not sure what it is... If there is no solution, does this mean once a ...
user avatar
  • 266
0 votes
2 answers
42 views

How to represent multi-label colours in one-hot encoding?

Say I want to predict the price of a gemstone based on its colour. I have two options: averaging over its colour on an RGB scale, or using its textual description. If I was to choose the latter, how ...
user avatar
0 votes
0 answers
15 views

How is the training comlexity of NNLM word2vec calculated?

I was reading this paper on word2vec, and came around the following description of a feedforward NNLM: It consists of input, projection, hidden and output layers. At the input layer, N previous words ...
user avatar
0 votes
1 answer
54 views

What is the difference between CNN-LSTM and RNN?

I'm starting to study RNN for a project of video prediction, but I encounter these CNN-LSTM models. Initially, I thought that is another name for RNN, but I think I get it wrong. Since I'm a beginner ...
user avatar
0 votes
0 answers
16 views

Neural Network with numerical data and sentences as features

I'm beginning in the words A.I. features. My current problem is that I want to create Neural Network that takes as input numerical data and also words as data (by words, I mean multiple sentences) to ...
user avatar
0 votes
0 answers
50 views

What is an attractor network?

Surprisingly, this wasn't asked before - at least there was one related question without any answers What is a continuous-attractor neural network?. So, what is an attractor network, and why should ...
user avatar
  • 101
2 votes
2 answers
78 views

Why are Siamese Neural Networks used instead of a single neural network?

Siamese Neural Networks are a type of neural network used to compare two instances and infer if they belong to the same object. They are composed by two parallel identical neural networks, whose ...
user avatar
0 votes
1 answer
30 views

Does make sense to add an additional Attention layer while Fine-Tuning Bert?

I'm fine tuning a Bert/Roberta model for a classification task. As I need to improve my results I'm thinking about to add an additional Attention layer after Bert Model and before Dense and Dropout ...
user avatar
3 votes
1 answer
60 views

Is it possible to train an AI to bring a picture story in the correct order (correct story flow)?

I want to know if it is possible to train a neural network (or some other kind of an AI) to bring a simple picture story in the correct order, if it is in random order, so that the story has the ...
user avatar
  • 33
1 vote
0 answers
40 views

How many layers do GPT-3, AlphaFold 2, and DALL-E 2 have?

Unsuccessfully, I tried to find out the "depth" (definition below) in large neural networks such as GPT-3, AlphaFold 2, and DALL-E 2. Formally, my question is about their computational graph:...
user avatar
0 votes
0 answers
24 views

What could be causing the poor performance (MSE) of a dense neural network on a real time-series dataset?

I am trying to understand time series analysis and actually I am following the course "Sequences, Time Series and Prediction" in Coursera. The course is based on a synthetic dataset, ...
user avatar
0 votes
0 answers
21 views

In NEAT, how do I prevent duplicate connections?

According to this paper, duplicate mutations are only given the same innovation numbers within the same generation. What do I do if a connection gets broken into 2 connections and a node during 2 ...
user avatar
1 vote
1 answer
119 views

What is the reason we loop over epochs when training a neural network?

After reading through this thread and some other resources online, I still do not understand the role of epochs in training a neural network. I understand that one epoch is one iteration through the ...
user avatar
1 vote
1 answer
37 views

Neural networks with sparse inputs

I have a task I want to solve with neural networks. The task is predicting a certain vector of dimension K. The problem is that the inputs to the networks are sparse. The input is a vector of size N, ...
user avatar
  • 121
0 votes
0 answers
23 views

Neural Network learning XOR. I collected Data on my networks convergence. Is this expected behavior?

I build a neural network from scratch to get a better understanding of the fundamentals of machine learning. The network contains a bias for each neuron and calculates the final error via the mean ...
user avatar
  • 101
0 votes
0 answers
12 views

Deep learning to fill sequence elements missing at random

I have the following problem setup: There is a list of floats (between -1 and 1) that is about 768*2 in length. The values of the floats are features that depend on two documents, the first 768 ...
user avatar
1 vote
1 answer
28 views

What type of ML or AI would predict personal traits from a DNA sequence?

Suppose you have a large dataset of DNA sequences. Alongside each sequence, you have a portrait of the person with the DNA sequence. Other parameters include the age, gender and race of the person. I ...
user avatar
  • 11
1 vote
1 answer
43 views

Is the initial teacher model in the Noisy Student algorithm noised?

Reading through the paper on the Noisy Student algorithm, I have a quick question about how the initial teacher model is built. In step 1 of the algorithm, the loss function is defined such that it ...
user avatar
0 votes
0 answers
51 views

What are the advantages and disadvantages of higher order neuron activation functions?

I've been reading about different types of neurons that the traditional linear one. One example I came across is the Sigma-Pi neuron, where the activation function includes higher order terms, such as ...
user avatar
0 votes
0 answers
21 views

Neural network have difficulty on capturing weak characteristics

I want use neural network to approximate a non-linear function. The function is, $$ F(X1,X2,X3) = A \times X1^{K1} \times exp((X1-X2) \times K2) \times exp(X3 \times K3) $$ where X1/X2/X3 are input ...
user avatar
0 votes
1 answer
20 views

Emulate program behavior using neural network

I have an exe file but no access to its source code. It takes as input a list of 8 parameters and prints text files containing the output. I was wondering if it is possible to write an AI-based ...
user avatar
0 votes
1 answer
58 views

When using TD(λ), how do you calculate the eligibility trace per input & weight of a neural network neuron?

I have a Neural Network, each Neuron is made up of inputs, weights, and output. I have potentially multiple hidden layers. The activation function executed against the output is not known by the ...
user avatar

1
2 3 4 5
44