Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
2
votes
0answers
12 views

Why does learning rate reduce train-test generalization gap?

In this blog post: http://www.argmin.net/2016/04/18/bottoming-out/ Prof Recht shows two plots: He says one of the reasons the plot below has a lower train-test gap is because that model was trained ...
1
vote
0answers
9 views

How do I trade-off samples versus specificity in deep-regression problems?

I'm working on a prediction problem where I have lots of data with natural hierarchies of specificity. An example of this hierarchy is in location data: e.g. Continent, country, city. With complete ...
1
vote
2answers
49 views

Should the training data be the same in each epoch?

Should the training data be the same in each epoch? If the training data is generated on the fly, for example, is there a difference between training 1000 samples with 1 epoch or training 1000 epochs ...
1
vote
1answer
22 views

Do correlations matter when building neural networks?

I am new to working with neural networks. However, I have built some linear regression models in the past. My question is, is it worth looking for features with a correlation to my target variable as ...
0
votes
1answer
34 views

Is the size of a neural network directly linked with an increase in its inteligence?

Just came across this article on GPT-3, and that lead me to the question: In order to make a certain kind of neural network architecture smarter all one needs to do is to make it bigger? Also, if that ...
1
vote
0answers
12 views

Could a quantum computer perform vectorized forward propagation in deep networks?

Forward propagation in Deep Neural Networks In the "Forward Propagation in a Deep Network" video on Coursera, Andrew NG mentions that there's no way to avoid a for loop to loop through the ...
1
vote
0answers
42 views

Why are these same neural network architecture giving different results?

I tried the first neural network architecture and the second one, but keeping all other variables constants, I am getting better results with the second architecture. Why are these same neural network ...
0
votes
1answer
39 views

Why isn't the loss of my neural network reduced after 2500 iterations?

I have developed a basic feedforward neural network from scratch to classify whether image is of cat or not cat. It works fine, but after 2500 iterations, my cost function is not reducing properly. ...
0
votes
1answer
42 views

Is my understanding of back-propogation correct?

I am trying to learn backpropagation and this is what I know so far. To update the weights of the neural network you have to figure out the partial derivative of each of the parameters on the loss ...
0
votes
0answers
27 views

Why is it that having a duplicate in features set makes training to work bad

I'm defining a deep network to emulate a multitarget regression. When I costruct my training set, I take information from a graph; without going into too much detail, it could happen that I take 2 ...
2
votes
0answers
22 views

Inaccurate masks with Mask-RCNN: Stairs effect and sudden stops

I've been using matterport's Mask R-CNN to train on a custom dataset. However, there seem to be some parameters that i failed to correctly define because on practically all of the images, the bottom ...
1
vote
1answer
18 views

When adding a feature is useless?

"simple" issue: I'm building a model, where from a feature set A I want to predict a target set C; I need to understand if another feature set B, together with A, can improve my model ...
2
votes
1answer
41 views

Which paper introduced the term “softmax”?

Nowadays, the softmax function is widely used in deep learning and, specifically, classification with neural networks. However, the origins of this term and function are almost never mentioned ...
2
votes
1answer
31 views

Should forecasting with neural networks only be treated as a supervised learning (regression) problem?

I have recently made a work about the application of neural networks to time series forecasting, and I treated this as a supervised learning (regression) problem. I have come across the suggestion of ...
1
vote
1answer
29 views

Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?

I am training a neural network that takes an input (H, W, 3) and has the output of size (H', W', C). Now, to augment my dataset, ...
1
vote
0answers
22 views

Isn't it true that using max over a softmax will be much slower because there is not a smooth gradient?

Isn't it true that using max over a softmax will be much slower because there is not a smooth gradient? Max basically zeros out the gradients of all the non-maximum values. Especially at the beginning ...
0
votes
0answers
34 views

Neural Network is not learning a very simple task

I am a complete beginner in the area. I implemented my first neural network following the online book "Neural Networks and Deep Learning" by Micheal Nielsen. It works fine with classifying ...
1
vote
1answer
19 views

How can I create an embedding layer to convert words to a vector space from scratch?

For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some ...
0
votes
0answers
10 views

Different result from k-cross validation model and Train-Validation-Test split model ? (AI fresher question)

I am starting to learn about Neural Network and I have come into one problem that I am still learning how to figure it out. I have a dataset with shape (105,96) (105 samples and 95 first column as ...
0
votes
0answers
9 views

Approaches for OCR building which can extract latex from the image as mathematical formulas

I have images of questions from the domain of mathematics, where the image can be a mixture of the English language and mathematical formulas. I want to build and train an OCR model like Harvard NLP's ...
0
votes
1answer
31 views

Does the input layer have bias and are there bias neurons?

I have seen two different representations of neural networks when it comes to bias. Consider a "simple" neural network, with just an input layer, a hidden layer and an output layer. To ...
0
votes
1answer
31 views

Input tensor shape order for RNN (PyTorch)

I am confused as to why the sequence length is the first dimension of the input tensor for an RNN when batch size is the first dimension for any other kind of network (Linear/CNN/etc.). This makes me ...
0
votes
0answers
21 views

Network design to learn multiple sequences of multiple categories

For learning a single sequence, LSTM only should suffice. However, my situation is different here. I have a list of sequences to learn: The sale volumes of 12 months, these are the sequences And ...
1
vote
2answers
49 views

Why the cost/loss starts to increase for some iterations during the training phase?

I am trying to build a recurrent neural network from scratch. It's a very simple model. I am trying to train it to predict two words (dogs and gods). While training, the value of cost function starts ...
1
vote
0answers
14 views

Suppress heatmap non-maxima in segmentation with UNet

I'm using U-Net for image segmentation. The model was trained with images that could contain up to 4 different classes. The train classes are never overlapping. The output of the UNet is a heatmap (...
0
votes
0answers
16 views

DQN Tic-Tac-Toe does not quite become optimal

I am trying to implement a DQN agent for playing standard 3x3 Tic-Tac-Toe (it is a double DQN with experience replay, and using a target network). I got the hyperparameters to the point where the ...
2
votes
0answers
45 views

Should batch normalisation be applied before or after ReLU?

I know that there has been some discussion about this (e.g. here and here), but I can't seem to find consensus. The crucial thing that I haven't seen mentioned in these discussions is that applying ...
0
votes
1answer
22 views

A neural network with 2 or more hidden layers is a DNN? [duplicate]

I just learnt the math behind neural networks so please bear with my ignorance. I wonder if there is a precise definition for DNN. Is it true that any neural network with more than 2 hidden layer can ...
0
votes
0answers
12 views

Looking for an Approach method for News maker

I have some databases with information about exports and imports of a country, what are they importing, the times that an importation was made, the country from and target, everything related to ...
0
votes
0answers
20 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how can Kneser-Ney Smoothing technique be integrated into a feed-...
0
votes
1answer
18 views

How is it possible to get the output size of `n` Consecutive Convolutional layers? [closed]

Given network architecture, what are the possible ways to define fully connected layer fc1 to have a generalized structure such as ...
1
vote
0answers
9 views

Algorithm to train a neural network against differentiable and non-differentiable databases?

Let's say I have two databases, $(\mathbf{x_i}, \mathbf{\hat{p_i}})$ and $(\mathbf{x_j}, \mathbf{\hat{q_j}})$. A neural network with weights $\theta$ can receive an input $\mathbf{x}$ and produce an ...
0
votes
0answers
23 views

Which loss function and evaluation metric should I use for a multiple output prediction problem?

I was running into a situation with a data set like this I have 4 events and and they might happen together in pairs. I want to use 3 features to predict the coupling between event. I am building a ...
0
votes
1answer
45 views

Why is non-linearity desirable in a neural network?

Why is non-linearity desirable in a neural network? I couldn't find satisfactory answers to this question on the web. I typically get answers like "real-world problems require non-linear ...
1
vote
1answer
35 views

Intuitively, why can the training of a neural network be formulated as a probability estimation problem?

Neural network training problems are oftentimes formulated as probability estimation problems (such as autoregressive models). How does one intuitively understand this idea?
1
vote
2answers
39 views

Why don't we use trigonometric functions for the output neurons?

Why don't we use a trigonometric function, such as $\tan(x)$, where $x$ is an element of the interval $[0,pi/2)$, instead of the sigmoid function for the output neurons (in the case of classification)?...
1
vote
1answer
26 views

Why does my “entropy generation” RNN do so badly?

I'm new to relatively RNNs, and I'm trying to train generative and guessing neural networks to produce sequences of real numbers that look random. My architecture looks like this (each "circle&...
0
votes
0answers
21 views

Reinforcement Learning Diagnostic: Total reward doesn't converge

I'm implementing DDQN in my toy scenario. During training, I'm surprised to see that the total reward doesn't converge and have a tendency to degrade. What could be the problem? Here's the picture: ...
1
vote
1answer
23 views

What's the best practice for Boltzmann Exploration temperature in RL?

I'm currently modeling DQN in Reinforcement Learning. My question is: what are the best practices related to Boltzmann Exploration? My current thoughts are: (1) Let the temperature decay through ...
0
votes
0answers
26 views

How do I interpret the following parameters as a network

I have a background in OR and I am new to AI. I know the basics and I currently try to understand the article "Learning Combinatorial Optimization Algorithms over Graphs". So far I ...
0
votes
2answers
16 views

Feature scaling strategy for many feature with very large variation between them?

I was running into a situation in which my input feature experience a very large variation in term of magnitude. Particularly, consider feature 1 belong to group 1 and feature 2 3 4 belong to group 2, ...
0
votes
0answers
5 views

Can attention with 2d position encoding beat capsule on cv tasks?

I have always had doubts about the necessity and intuitive/theoretical justification for capsule network in image classification and more recently nlp tasks. For the former, in order to address the ...
1
vote
2answers
88 views

Do convolutional neural networks perform convolution or cross-correlation?

Typically, people say that convolutional neural networks (CNN) perform the convolution operation, hence their name. However, some people have also said that a CNN actually performs the cross-...
0
votes
0answers
25 views

How to train neural network a math multiplication table? [migrated]

I am trying to train neural network (brain.js) a multiplication table. It is not going too well: requires lots of hidden layers, iterations and very small error threshold, and the results are still ...
0
votes
2answers
35 views

How to combine several chatbots into one?

I'm in the middle of a project in which I want to generate a TV series script (characters answering to each other, scene by scene) using SOTA models, and I need some guidance to simplify my ...
0
votes
1answer
62 views

What is a convolutional neural network?

Given that this question has not yet been asked on this site, although similar questions have already been asked in the past (e.g. here or here), what is essentially a convolutional neural network (...
1
vote
1answer
42 views

How do GPUs faciliate the training of a Deep Learning Architecture?

I would love to know in detail, how exactly GPUs help, in technical terms, in training the deep learning models. To my understanding, GPUs help in performing independent tasks simultaneously to ...
0
votes
2answers
84 views

When we use a neural network to approximate the Q values, is the Q target a single value?

I have two questions When we use our network to approximate our Q values, is the Q target a single value? During backpropagation, when the weights are updated, does it automatically update the Q ...
0
votes
0answers
53 views

TD-Leaf struggles at learning chess

I am currently working on implementing Giraffe chess algorithm. Following this paper, I designed a neural network similar to the one proposed by the author which I trained using TD-Leaf(lambda). The ...
1
vote
0answers
23 views

Why can't neural networks be applied to preference learning problems?

In section 6.1 of the paper Neural Networks in Economics, the authors say this leads to the problem, that no risk can be formulated which shall be minimized by a Neural Network learning algorithm. ...

1
2 3 4 5
34