Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
1
vote
1answer
27 views

In NN, as iterations of Gradient descent increases, the accuracy of Test/CV set decreases. how can i resolve this?

As mentioned in the title I'm using 300 Dataset example with 500 feature as an input. As I'm training the dataset, I found something peculiar. Please look at the data shown below. Iteration 5000 |...
0
votes
0answers
25 views

I am looking for research related to the use of AI and ML in car, aeronautics manufacturing design and safety

I am specifically interested in the topic of edge cases. I have the presentation Edge Cases and Autonomous Vehicle Safety as a starting point, in particular on page 6: Machine Learning (inductive ...
2
votes
1answer
45 views

How do I know if my dataset is ready for a machine learning model?

I am new in this area of Machine Learning and Neural Networks. Currently, I'm taking some courses on Udemy and reading a book about it, but I still have one big question regarding data pre-processing. ...
2
votes
0answers
41 views

Designing state representation for board game

I am trying to write self-play RL (NN + MCTS http://web.stanford.edu/~surag/posts/alphazero.html) to "solve" a board game. However, I got stuck in designing boardgame same (input layer for NN). 1) ...
1
vote
2answers
58 views

How to create neural network that predicates result of exam?

Actually, I am "fresh-water", and I've never known what is neural network. Now I am trying understand how to design simple neuronetwork for this problem: I'd like to make up such neural network that ...
0
votes
1answer
37 views

Analysis of Training Loss and Validation Loss Graph

Here I am Showing Two Loss graphs of an Artificial Neural Network. Model 1 Model 2 Blue -training loss Red -val training loss Can you help me to analyse these graphs? I read some articles and ...
0
votes
1answer
34 views

Decide Number of input Parameters and Output Parameters - ANN

I have to create a Neural Network for regression purpose. Basically, I created a Model which predict next 5 values when we give past 6 values. I want to make a change in this neural network. For ...
0
votes
1answer
30 views

Is there any way to classify Document Image without OCR?

I have multiple invoices images which need to classify invoice types such as fright, utility, goods, etc. Is there any way to classify without OCR?
2
votes
1answer
21 views

How to add variation in the results of a neural networks?

I would like to create a neural network that converts text into handwriting for use with a pen plotter. Before I start on this project, I'd like to be sure that artificial intelligence is the best way ...
0
votes
1answer
46 views

Back propagation on matrix of weights

I am trying to implement a Neural Network for binary classification using python and numpy only. My network structure is as follows: input features: 2 [1X2] matrix Hidden layer1: 5 neurons [2X5] ...
1
vote
0answers
51 views

Neural networks when gradient descent is not possible

I am looking for an example in which it is simply impossible to use some sort of gradient descent to train a neural network. Is this available? I have read quite some papers about gradient-free ...
0
votes
1answer
15 views

How to handle proper names or variable names in word2vec?

The input in word2vec is known word (spellings), each tagged by its ID. But if you process real text, there can be not only dictionary words but also proper nouns like human names, trade marks, file ...
4
votes
0answers
89 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
2
votes
1answer
26 views

Changes in flow detection neural network?

Do you have any advice, what architecture of neural network is the best for following task? Let input be some (complex function), the neural network gains a flow of its values, so I guess there will ...
1
vote
0answers
12 views

Questions regarding rrn-writer by Robin Sloane?

https://github.com/robinsloan/rnn-writer I preface this by saying I do not know much about this topic, only that I have an intense interest in it, so I'm hoping I can make my questions as clear as ...
1
vote
1answer
25 views

Further Normalization of Standardized data - ANN

I want to develop a regression model using the artificial neural network. For developing such a model I use standardised ( z-score normalised ) data. given below is the sample data set. Here MAX is ...
2
votes
2answers
102 views

What is the use of softmax function in a CNN?

What is the use of softmax function? Why was it used at the end of fully connected layer in convolution neural network?
0
votes
0answers
22 views

How to use TensorFlow with hyperparameter tuning to optimize parameters for a robot simulator

I am trying to implement a DNN to optimize a set of 7 parameters that are used in a robot swarm simulator on the ARGoS platform. the program is a compiled C++ executable that reads the parameters from ...
3
votes
1answer
80 views

How is G(z) related to x in GAN proof?

In the proofs for the original GAN paper, it is written: $$∫_x p_{data}(x) \log D(x)dx+∫_zp(z)\log(1−D(G(z)))dz =∫_xp_{data}(x)\log D(x)+p_G(x) \log(1−D(x))dx$$ I've seen some explanations asserting ...
0
votes
0answers
24 views

Why such a big difference in number between training error and validation error?

Question Why such a big difference between my 'Train loss' and 'Validation loss' as shown in the picture below? Is it a signal that my codes are wrong and my trained network is wrong as well? Some ...
0
votes
0answers
21 views

Weak gradient around one hot values of softmax

If you do the math for the softmax gradient, the gradient is very weak around the simplexical vertices. Aka for a 5-class softmax,[1, 0, 0, 0, 0] has a hard time moving to [0, 1, 0, 0, 0]. I have this ...
2
votes
0answers
38 views

Neural Network for Error Prediction of a Physics Model?

I have physical model prediction data as well as actual data. From this I can calculate the error of each prediction data point through simple subtraction. I am hoping to train a neural network to be ...
2
votes
1answer
33 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
1
vote
1answer
40 views

Do I need to use a pre-processed dataset to classify comments?

I want to use Machine Learning for text classification, more precisely, I want to determine whether a text (or comment) is positive or negative. I can download a dataset with 120 million comments. I ...
2
votes
1answer
56 views

How do I classify an image that contains only polygons?

I have two closed polygons, drawn as connected straight black lines on a white background. I need to classify such images in to three forms Two separate polygons One polygon encloses the other The ...
0
votes
0answers
25 views

DCGAN loss determining data normalization problems

I'm working with a DCGAN, a deep CNN for classifying images with a GAN that competes with the classifier to generate images of what we are classifying. The goal of the project at the moment is to ...
1
vote
0answers
40 views

RNN weights when varying the input size

I have a time-varying input size vector for a RNN. However, I am facing some difficulties understanding how to deal with my network weights when the input changes. Say we have a set of natural ...
2
votes
2answers
39 views

Balancing learning and structure in a neural network

In general, how does one balance the two opposing forces of allowing a layer of a neural network to adapt/learn to its training data, while also forcing the neural network to represent some known ...
2
votes
0answers
17 views

Validation Loss Fluctuates then Decrease alongside Validation Accuracy Increases

I was working on CNN. I modified the training procedure on runtime. As we can see the validation loss and validation accuracy, The yellow curve does not fluctuate much. Green curve and Red curve ...
1
vote
0answers
14 views

How important is architectural similarity between the discriminator and generator of a GAN?

Shouldn't the discriminator and generator work fine even if they don't process data symmetrically? I mean, they don't only receive the final layer results of each other, they don't use data that from ...
1
vote
1answer
38 views

How to deal with invalid output in a policy network? [duplicate]

I am interested in creating a neural network-based engine for chess. It uses a $8 \times 8 \times 73$ output space for each possible move as proposed in the Alpha Zero paper: Mastering Chess and Shogi ...
1
vote
1answer
37 views

Why is expectation used instead of simple sum in GANs?

Why do GAN loss functions use expectation(sum + division) instead of a simple sum?
0
votes
0answers
51 views

Updating biasses while backpropagation in all details

I just wabt to ask do i understand the usage and updating biasses while backpropagating and what values comes where. Let us see following network : with bach size 6, input size 2, hidden size 4 and ...
2
votes
0answers
16 views

In addition to matrix algebra, can GPU's also handle the various Kernel functions for Neural Networks?

I've read a number of articles on how GPUs can speed up matrix algebra calculations, but I'm wondering how calculations are performed when one uses various kernel functions in a neural network. If ...
1
vote
1answer
44 views

Are there neural networks that accept graphs or trees as inputs?

As far I know, the RNN accepts a sequence as input and can produce as a sequence as output. Are there neural networks that accept graphs or trees as inputs, so that to represent the relationships ...
3
votes
3answers
75 views

How does adding a small change to an neuron's weighted input affect the overall cost?

I was reading the following book: http://neuralnetworksanddeeplearning.com/chap2.html and towards the end of equation 29, there is a paragraph that explains this: However I am unsure how the ...
2
votes
2answers
55 views

Returning function that a neural network estimated

As stated in the Universal approximation theorem, a neural network can approximate almost any function. I was wondering, if there are methods to return the actual formula of the function that the NN ...
1
vote
2answers
94 views

Neural nets for novices

Stories like this one are quite popular these days. The idea of training a neural net to do something silly like this may sound trivial to experts like you, but for a novice like me it could be an ...
0
votes
0answers
47 views

Convolutional neural network debugging

Im trying to implement CNN for small images classification (36x36x1) (grayscale). I've checked every forward/backward pass function on small example, and still my cnn is not doin any progress on ...
0
votes
0answers
55 views

Super Resolution on text documents

I want to implement super-resolution and deblurring on images from text documents. Which is the best approach? Are there any Git-hub links which will help me to start? I am new to the field. Any help ...
0
votes
1answer
23 views

What type of network for a repeated experiment

good day I have a problem where i have 9 data points that are collected every minute for 40 minutes, and by the 40th minute the solution would be either end up being black or white. I would like to ...
0
votes
0answers
37 views

How to learn to sample?

Imagine you have access to a dataset of pairs $(s, \hat{\pi}(s))$ where s is a state in a high dimension continuous space $S$, $\pi(s)$ is a probabilistic distribution on a large discrete space $D$ (...
1
vote
2answers
95 views

Is it possible to train an animal so that it becomes as intelligent as a human?

The human and animal brain is made of neural networks. Is it possible to train an animal so that it becomes as intelligent as a human?
1
vote
0answers
21 views

How do GANs create an image with specific characteristics?

I've seen GANs that do things like convert an image to a painting or this GAN here https://make.girls.moe/#/ that takes in a set of characteristics and generates a waifu with those characteristics. ...
5
votes
4answers
271 views

Is it possible for a neural network to be used to compress data?

When training a neural network, we often run into the issue of overfitting. However, is it possible to put overfitting to use? Basically, my idea is, instead of storing a large dataset in a database, ...
1
vote
0answers
27 views

One end to end Neural network or many task-specific ones?

Is it better to train one neural network for a dispersed labeled data with large number of classes or first classify data by unsupervised learning then train each part by a separate NN? I mean by ...
0
votes
0answers
50 views

A2C Critic Loss Interpretation

I'm working on an Advantage A2C implementation, and I just finished creating the value network $\hat{V_{\phi}}$. I train this network with the standard MSE loss of discounted rewards-to-go:$$\|\hat{V_{...
2
votes
1answer
42 views

Convolutional layer to Fully Connected Layer implementation

Im implementing a neural network framework from scratch in C++ as a learning exercise. There is one concept I don't see explained anywhere clearly: How do you go from your last convolutional/pooling/...
8
votes
2answers
828 views

Is neural networks training done one-by-one?

I'm trying to learn neural networks by watching this series of videos and implementing a simple neural network in Python. Here's one of the things I'm wondering about: I'm training the neural network ...
1
vote
2answers
118 views

Why are VAE's useful?

I am not sure I understand what is the advantage of using a VAE's over a deterministic Auto Encoder? For example, assuming we have just 2 labels, a deterministic Auto Encoder will always map a given ...