Questions tagged [neural-networks]
For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.
699
questions with no upvoted or accepted answers
2
votes
1answer
246 views
Unable to overfit using MLP
I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
2
votes
0answers
551 views
Getting worse performance when training a pre-trained model with the existing class
I am training pre-trained SSD-InceptionV2-Coco to detect the "car",
which is one of the classes in mscoco label.
I train the model with ~50k sample from KITTI, 500k iteration with batch size 2.
I ...
2
votes
0answers
369 views
Coding CGAN paper model in Keras
I was coding a CGAN model using Keras along with the paper (https://arxiv.org/pdf/1411.1784.pdf) and I wanted to try and match the models to exactly what the paper says. I knew the models presented in ...
2
votes
0answers
125 views
Classification Learning - Normalization of time series and live usage
UPDATE: The tables look messed up so i put them on pastebin for better
visibility. https://pastebin.com/gDX28uVF
I am using Neural Network with different learning types (for example Standard ...
2
votes
0answers
90 views
NLP proved against US legal texts
I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts.
Obviously some training would need to be done, but I've found little ...
2
votes
0answers
41 views
How to apply EOT algorithm to 3d model
Many of you have probably seen the turtle from LabSix that gets mistaken for a rifle in Google's InceptionV3 image classifier. I read the paper and I understand how they apply EOT to 2d images and on ...
2
votes
0answers
44 views
Condition Action Statement - Feed Forward Neural Network
I am trying to produce Decision Tree from Feed Forward Neural Network .
The input to the feed forward neural network is Condition Action Statement
for example, if airthrusthold > 90 , power up the ...
2
votes
0answers
132 views
Data prepared to linear regression. Can I use it with backpropagation?
I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the Java Neural Network Simulator, JavaNNS, program.
In one practice I have to build a neural network to use ...
2
votes
0answers
128 views
Stacked softmax layers before output
I have seen people using stacked softmax layers right at the output of neural networks designed for classification. I'm trying to understand this. Does it give any additional value? I think this could ...
2
votes
0answers
265 views
Tensorflow: Can't overfit training data with batch size > 1
I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
2
votes
0answers
78 views
Algorithms that connect neurons to previous layers as well as next
Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes ...
2
votes
0answers
50 views
Mini (local) NN for Mobile?
I'm working on architecture for a game AI where, due to the nature of the game, the classical approach seems likely be sufficient to beat most humans--the endgame is tractable and traditional game-...
2
votes
0answers
168 views
Help with implementing Q-learning for a feedfoward network playing a video game
I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives.
In ...
2
votes
0answers
138 views
Wide & Deep Learning Explanation
I was going through "Wide & Deep Learning" tensorflow tutorial & it's quite simply explained the process. But I missed few of the things. If someone can please explain them to me, it will be ...
2
votes
0answers
47 views
Recommendations on which architecture to use to guess appointment
I'm currently developping an application which allows psychologists to manage their schedule and budget. As a proof of concept, I would like to create an intelligent appointment service. There can be ...
2
votes
0answers
172 views
Is iLQG a good algorithm for model-based planning with simple environments?
In their work Continuous Deep Q-Learning with Model-based Acceleration, the author demonstrate great results of applying Imagination Rollouts for model-based acceleration of learning. They test their ...
2
votes
0answers
45 views
Loss of precision when encoding DNN weights
This question is related to the usage of NN in critical systems (those where a failure can cause life threatening situations - autopilots for example) and the need for formal guarantees on their ...
2
votes
0answers
208 views
Can anybody explain such behavior of accuracy and loss of my Net(caffe)?
I used this project for example(framework - caffe, arhitecture of net - mod of AlexNet, 400 images are used for training). I have this result:
or this:
Solver:
...
2
votes
0answers
119 views
Train, Validation and Test Split for Reporting Accuracy of Neural Model and BOW
I need to report accuracies of my neural model in a conference paper as compared to various baselines. What are the accepted standards for reporting accuracies in a fair manner?
Neural Model:
To be ...
2
votes
0answers
70 views
seq2seq vector to letters model
I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
1answer
148 views
How do I classify an image that contains only polygons?
I have two closed polygons, drawn as connected straight black lines on a white background. I need to classify such images in to three forms
Two separate polygons
One polygon encloses the other
The ...
2
votes
1answer
117 views
Why do the inputs and outputs of a convolutional layer usually have the same depth?
Here's the famous VGG-16 model.
Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth? What's the reason for that?
Is there a theory or paper trying to ...
2
votes
1answer
213 views
Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?
I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e....
1
vote
0answers
28 views
Are monotonically increasing functions easier to learn?
A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant.
Logically this seems like ...
1
vote
1answer
46 views
How to extract parameters from a text using AI/NLP
lets say I have three texts:
"make a heading that says hello word"
"make a heading of hello world"
"create heading consist of hello world"
How can I fetch those groups ...
1
vote
0answers
13 views
How can an “architectural motif” be extracted from a trained MLP?
I am trying to reproduce the paper Synthetic Petri Dish: A novel surrogate model for Rapid Architecture Search. In the paper, the authors try to reduce the architecture of an MLP model trained on ...
1
vote
0answers
17 views
For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?
When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations.
For example, here (slide 20), given $o$ (the output, defined in slide 18), $z$ (the ...
1
vote
0answers
18 views
Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?
I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
1
vote
0answers
14 views
Drawing verticle lines on video frames in vehicle detection
I am working on an application of vehicle detection, the purpose is to check if road is congested or not. The solution that came to my mind is to detect the vehicles moving in each direction ...
1
vote
0answers
32 views
Designing Policy-Network for Deep-RL with Large, Variable Action Space
I am attempting a project involving training an agent to play a game using deep reinforcement learning.
This project has a few features that complicate the design of the neural network:
The action ...
1
vote
0answers
24 views
The last target name is missed in the test set
I am training a neural network with a dataset that has 51 classes and 6766 data in it. I used 80% for the training set, 10% for validation, and 10% for the test. After training I got confusion matrix ...
1
vote
0answers
57 views
Are Graph Neural Networks generalizations of Convolutional Neural Networks?
In lecture 4 of this course, the instructor argues that GNNs are generalizations of CNNs, and that one can recover CNNs from GNNs.
He presents the following diagram (on the right) and mentions that it ...
1
vote
0answers
21 views
Papers on using symbolic methods as constraint on neural network?
Given a set of constraints on the input data, I am looking for papers that discuss using symbolic methods (decision trees, rule based, etc.) as a separate source of certainty in a classification task. ...
1
vote
0answers
21 views
Building a resume recommendation for a job post?
There are few challenges I am facing when building a resume recommendation for a particular job positing.
Let's say we convert the resume into a vector on n-dimensions and job description also as an n-...
1
vote
0answers
58 views
Is the performance of a neural network, which was trained with encrypted data and weights, affected if the weights are decrypted?
Suppose that a neural network is trained with encrypted (for example, with homomorphic encryption and, more precisely, with the Paillier partial scheme) data. Moreover, suppose that it is also trained ...
1
vote
0answers
21 views
How do I infer exploding or vanishing gradients in Keras?
It may already be obvious that I am just a practitioner and just a beginner to Deep Learning. I am still figuring out lots of "WHY"s and "HOW"s of DL.
So, for example, if I train a ...
1
vote
0answers
16 views
How does noise input size affect fake image generation with GANs?
In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. ...
1
vote
0answers
34 views
What is the gradient of an attention unit?
The paper Attention Is All You Need describes the Transformer architecture, which describes attention as a function of the queries $Q = x W^Q$, keys $K = x W^K$, and values $V = x W^V$:
$\text{...
1
vote
0answers
15 views
Looking for a good approach for building an automated director for a racing game spectator mode
I'm building a tool that should assist a director to broadcast a racing game. I want this tool to suggest the human director which car to focus on and with which camera (among the available ones). I ...
1
vote
0answers
25 views
Why should variance(output) equal variance(input) in Xavier Initialisation?
In a lot of explanations online for Xavier Initialization, I see the following:
With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a ...
1
vote
0answers
21 views
Neural network architecture with inputs and outputs being an unkown function each
I am trying to set up a neural network architecture that is able to learn the points of one function (blue curves) from the points of an other one (red curves). I think that it could be somehow ...
1
vote
0answers
16 views
React on train-validation curve after trening
I have a regression task that I tray to solve with AI.
I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance)
I understand basic ...
1
vote
0answers
52 views
How does NN follows law of energy conservation?
Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average ...
1
vote
0answers
24 views
How does Google's 2016 GNMT architecture work?
I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me.
This is a picture ...
1
vote
0answers
34 views
How to use unmodified input in neural network?
My question may be a bit hard to explain...
My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2.
From this ...
1
vote
0answers
43 views
What are the use-cases of self-replicating neural networks?
I started researching the subject of self-replication in neural networks, and unexpectedly I saw that there is not much research on this subject. I should mention I am new in the field of NNs.
This ...
1
vote
0answers
15 views
Effect of Batch Size on a Translation Model's Validation Scores
This is a general question.
I am training a neural machine translation model with different batch sizes: 16, 32, and 64 for the same amount of epochs (10).
I am getting very low validation scores for ...
1
vote
0answers
39 views
Where can I find pre-trained agents able to play games with multiple stages like exploration, dialog, combat?
My goal is to create an ML model to be able to classify different game stages, e.g., dialog with a non-player character, exploration, combat with enemy, in-game menu etc.
In order to do that, I am ...
1
vote
0answers
33 views
In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?
While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the ...
1
vote
0answers
17 views
How does scheduled sampling for transformers work?
I was reading this paper which applies a modified version of the transformers for traffic forecasting. I am somewhat familiar with the transformer architecture and how it functions, but, in the paper, ...
