Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Is there a rule-of-thumb to determine which behaviours must be learned in a lifetime and which innate?

I was training an AI to learn things during its lifetime such as find food and navigate a maze. Behaviors that might change during its lifetime. But I hit upon a snag. Some behaviors, like avoiding ...
1
vote
1answer
9 views

Why are activation functions independent layers in CNNs rather than part of convolutional layers?

I have been reading up on CNNs. One of the different confusing things has been that people always talk of normalization layers. A common normalization layer is a ReLU layer. But I never encountered an ...
1
vote
2answers
17 views

Are feature maps merged or are they passed on as they are?

I am unsure about the following parts of the architecture and mechanics of convolution layers in CNNs. Possibly, this is implementation-dependent though. First question: Say I have 2 convolution ...
2
votes
0answers
23 views

Will the RL agent implemented as a neural network fine-tune itself?

Normally, when you develop a neural network, train it for object recognition (on normal objects like bike, car, plane, dog, cloud, etc.), and it turns out to perform very well, you would like to fine-...
1
vote
1answer
34 views

How do layers in an artificial neural network transform inputs to outputs?

To me, most ANN/RNN related articles don't tell me actually how the network is implemented. I know that in the ANN you'll have multiple neurons, activation function, weights, etc. But, how do you, ...
2
votes
1answer
40 views

Why do we normalize data in a deep neural network?

I have asked this question a number of times, but I always get confusing answers to this, like "normalized data works better", "data lives in the same scale" How can ...
2
votes
1answer
25 views

Is a multi-layer Kohonen network possible?

The Kohonen network is one fully connected layer, which clusters the input into classes by a given metric. However, the one layer does not allow to operate with complex relations, that's why deep ...
0
votes
1answer
22 views

How can I perform multivariable regression with neural networks?

I want to use a neural network to perform a multivariable regression, where my dataset contains multiple features, but I can't for the life of me figure it out. Every kind of tutorial on the internet ...
0
votes
1answer
27 views

Why does keras model get bigger after training?

I notice that I create a model using tensorflow.keras.Sequential(), save it and the file size is around 5 MiB, but after I call ...
1
vote
1answer
17 views

Better to learn the same small set for multiple epochs then go to the next or learn from each one time repeatedly for multiple times?

I don't know if I worded the title correctly. I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save ...
1
vote
1answer
35 views

Dropout causes too much noise for network to train

I am using dropout of different values to train my network. The problem is, dropout is contributing almost nothing to training, either causing so much noise the error never changes, or seemingly ...
1
vote
0answers
53 views

What can be inferred about the training data from a trained neural network?

Suppose we trained a neural network on some training set that we call $X$. Given the neural network and the method of training(algorithm, hyperparameters etc.) can we infer anything about $X$. Now, ...
1
vote
0answers
57 views

How do I determine the generalisation ability of a neural network?

I am trying to ascertain if my neural network is able to generalize or if it’s simply using memory/overfitting to solve a task. I would like my model to generalise. Currently, I train the neural ...
0
votes
0answers
37 views

Encoding real valued inputs

I have the following issue: I want to train a network (used in a variation of deep q learning that I use for a pricing decision) to predict me the value of a certain state/action combination. The ...
4
votes
1answer
80 views

Where can I find the proof of the universal approximation theorem?

The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
0
votes
0answers
32 views

Confused about NeuralODE

I am a bit confused about NeuralODE and I want to make sure that what I understood so far is correct. Assume we have (for simplicity) 2 data points $z_0$ measured at $t_0$ and $z_1$ measured at $t_1$...
0
votes
0answers
28 views

Why following GAN is not working:

Trying to see, why my big GAN project is not working, i created the small GAN project to see interaction of generator and discriminator. Dummy input is like: ...
0
votes
0answers
16 views

Why epsilon-greedy hyperparameter is annealed smoothly?

Regarding of DQN, or DQRNN, (reinforcement learning) To me, RL is a process that can be divided into 2 stages: Exploring wide range of paths (acting randomly) Refining the current optimal paths (...
1
vote
1answer
55 views

Is there an AI model with “certainty” built in?

If I see a hundred elephants and fifty of them are grey I'd say the probability of an elephant being grey is 50%. And my certainty of that probability is high. However, if I see two elephants and one ...
36
votes
4answers
9k views

Are neural networks prone to catastrophic forgetting?

Imagine you show a neural network a picture of a lion 100 times and label with "dangerous", so it learns that lions are dangerous. Now imagine that previously you have shown it millions of images of ...
0
votes
1answer
33 views

Is it possible with stochastic gradient descent for the error to increase?

As simple as that. Is there any scenario where the error might increase, if only by a tiny amount, when using SGD (no momentum)?
2
votes
0answers
39 views

Is a neural network the correct approach to optimising a fitness function in a genetic algorithm?

I've written an application to help players pick the optimal heroes during the draft phase of the Heroes of the Storm MOBA. It can be daunting to pick from 80+ characters that have synergies/counters ...
1
vote
0answers
60 views

Is this a classification problem?

I’m not really sure which machine learning approach is best for my problem at hand. I work in an engineering company that designs and builds different kinds of ships. In my particular job, I collect ...
3
votes
1answer
121 views

Can a vanilla neural network theoretically achieve the same performance as CNN?

I perfectly understand that CNN takes into account the local dependency of each pixel to the nearby pixels. In addition, CNNs are spatially invariant which means that they are able to detect the same ...
1
vote
0answers
26 views

How to calculate size and offset of YOLO grid in a fully convolutional network with zero padding? [migrated]

Fully convolutional network with zero padding: I have a fully convolutional network which does not have any padding in convolutional layers. This implies that, after each convolution operation, the ...
1
vote
1answer
39 views

Which loss function should I use for binary classification?

I plan to create a neural network using Python, Keras and TensorFlow. All the tutorials I have seen so far are concerned with image recognition. However, the goal of my program would be to take in 10+ ...
1
vote
0answers
18 views

Disabling of genes during crossover (NEAT)

I am implementing NEAT (Neuroevolution of augmenting topologies) by Stanley, Original Paper. I am facing a problem during crossover of genomes. Suppose two networks with connections ...
0
votes
1answer
33 views

Is there a place where I can read or watch to get an accurate TensorFlow code wise explanation?

I have a piece of code and I don't seem to really understand it but I'd love to get a source/link/material that would help me understand the basic functions in TensorFlow. Are there any recommended ...
1
vote
0answers
32 views

Is it possible to use adversarial training to learn invariant features?

Given a set of time series data that are generated from different sites where all sites are investigating the same objective but with slightly different protocols. Is it possible to use adversarial ...
1
vote
1answer
63 views

Reinforcement learning to play snake - network seems to not get trained at all

I am trying to build a network able to play snake game. This is my very first attempt to do such stuff. Unfortunately, I've stuck and even have no idea how to reason about the problem. I use ...
2
votes
0answers
36 views

Why do neural networks have bias units?

Why do neural networks have bias units? Why is it sometimes okay to opt them out?
1
vote
2answers
60 views

How can I train a neural network to give probability of a random event?

Let's say I have an adjustable loaded die, and I want to train a neural network to give me the probability of each face, depending on the settings of the loaded die. I can't mesure its performance on ...
0
votes
0answers
41 views

Why doesnt my lstm model for time series prediction improve after certain level of performance?

I created an lstm model which predicts multioutput sequeances. It takes variable length sequences as input. These sequences are padded with zero to obtain equal length. Note that the time series are ...
1
vote
1answer
43 views

Is there a neural network method for time-varying directed graphs?

I want to study NN for time-varying directed graphs. However, as this field has developed relatively recently, it is difficult to find new ways. So the question is, is there any NN that can handle ...
1
vote
0answers
48 views

Why is graph convolution network in time-varying graphs useful for anomaly detection?

In this paper, the authors refer to the application of time-varying graphs as an open problem. And they say it will be useful for anomaly detection in financial networks, etc. But why is that useful?
1
vote
1answer
29 views

How does the CTC loss work?

I am trying to implement CTC loss in Tensorflow, but their documentation is pretty limited. So I am not sure how to approach the problem. I found a good example in Theano: https://github.com/...
0
votes
0answers
12 views

L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks

Could anyone help to derive equation (15) dL/dx in L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks ?
1
vote
0answers
21 views

Is there any useful source on High Bias vs High variance issue on Neural Network?

I've been struggling to analyize my NN model. I've studied through andrew ng's course, but there are some results that cannot be explained by the course. Is there any useful source on High Bias vs ...
1
vote
0answers
11 views

What are some neural network models that can use auxiliary info during training for image segmentation?

What are some deep learning models that can use supplementary information other than RGB channels for image segmentation? For example imagine a poorly shot image of a river (blue) that shows a gap, ...
1
vote
1answer
22 views

In NN, as iterations of Gradient descent increases, the accuracy of Test/CV set decreases. how can i resolve this?

As mentioned in the title I'm using 300 Dataset example with 500 feature as an input. As I'm training the dataset, I found something peculiar. Please look at the data shown below. Iteration 5000 |...
0
votes
0answers
23 views

I am looking for research related to the use of AI and ML in car, aeronautics manufacturing design and safety

I am specifically interested in the topic of edge cases. I have the presentation Edge Cases and Autonomous Vehicle Safety as a starting point, in particular on page 6: Machine Learning (inductive ...
2
votes
1answer
41 views

How do I know if my dataset is ready for a machine learning model?

I am new in this area of Machine Learning and Neural Networks. Currently, I'm taking some courses on Udemy and reading a book about it, but I still have one big question regarding data pre-processing. ...
2
votes
0answers
35 views

Designing state representation for board game

I am trying to write self-play RL (NN + MCTS http://web.stanford.edu/~surag/posts/alphazero.html) to "solve" a board game. However, I got stuck in designing boardgame same (input layer for NN). 1) ...
1
vote
2answers
54 views

How to create neural network that predicates result of exam?

Actually, I am "fresh-water", and I've never known what is neural network. Now I am trying understand how to design simple neuronetwork for this problem: I'd like to make up such neural network that ...
-1
votes
0answers
15 views

Test Results from Synaptic Formula. Beginnings. Optimizing Neural Networks. Is this NP Complete?

(base) C:\Users\HououinKyouma>python Python 2.7.16 |Anaconda, Inc. Type "help", "copyright", "credits" or "license" for more information. import os import numpy as np import ...
0
votes
1answer
29 views

Analysis of Training Loss and Validation Loss Graph

Here I am Showing Two Loss graphs of an Artificial Neural Network. Model 1 Model 2 Blue -training loss Red -val training loss Can you help me to analyse these graphs? I read some articles and ...
0
votes
1answer
31 views

Decide Number of input Parameters and Output Parameters - ANN

I have to create a Neural Network for regression purpose. Basically, I created a Model which predict next 5 values when we give past 6 values. I want to make a change in this neural network. For ...
0
votes
1answer
24 views

Is there any way to classify Document Image without OCR?

I have multiple invoices images which need to classify invoice types such as fright, utility, goods, etc. Is there any way to classify without OCR?
2
votes
1answer
17 views

How to add variation in the results of a neural networks?

I would like to create a neural network that converts text into handwriting for use with a pen plotter. Before I start on this project, I'd like to be sure that artificial intelligence is the best way ...
0
votes
1answer
42 views

Back propagation on matrix of weights

I am trying to implement a Neural Network for binary classification using python and numpy only. My network structure is as follows: input features: 2 [1X2] matrix Hidden layer1: 5 neurons [2X5] ...