Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

699 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2
votes
1answer
246 views

Unable to overfit using MLP

I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
2
votes
0answers
551 views

Getting worse performance when training a pre-trained model with the existing class

I am training pre-trained SSD-InceptionV2-Coco to detect the "car", which is one of the classes in mscoco label. I train the model with ~50k sample from KITTI, 500k iteration with batch size 2. I ...
2
votes
0answers
369 views

Coding CGAN paper model in Keras

I was coding a CGAN model using Keras along with the paper (https://arxiv.org/pdf/1411.1784.pdf) and I wanted to try and match the models to exactly what the paper says. I knew the models presented in ...
2
votes
0answers
125 views

Classification Learning - Normalization of time series and live usage

UPDATE: The tables look messed up so i put them on pastebin for better visibility. https://pastebin.com/gDX28uVF I am using Neural Network with different learning types (for example Standard ...
2
votes
0answers
90 views

NLP proved against US legal texts

I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts. Obviously some training would need to be done, but I've found little ...
2
votes
0answers
41 views

How to apply EOT algorithm to 3d model

Many of you have probably seen the turtle from LabSix that gets mistaken for a rifle in Google's InceptionV3 image classifier. I read the paper and I understand how they apply EOT to 2d images and on ...
2
votes
0answers
44 views

Condition Action Statement - Feed Forward Neural Network

I am trying to produce Decision Tree from Feed Forward Neural Network . The input to the feed forward neural network is Condition Action Statement for example, if airthrusthold > 90 , power up the ...
2
votes
0answers
132 views

Data prepared to linear regression. Can I use it with backpropagation?

I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the Java Neural Network Simulator, JavaNNS, program. In one practice I have to build a neural network to use ...
2
votes
0answers
128 views

Stacked softmax layers before output

I have seen people using stacked softmax layers right at the output of neural networks designed for classification. I'm trying to understand this. Does it give any additional value? I think this could ...
2
votes
0answers
265 views

Tensorflow: Can't overfit training data with batch size > 1

I coded a small RNN network with Tensorflow to return the total energy consumption given some parameters. There seem to be a problem in my code. It can't overfit the training data when I use a batch ...
2
votes
0answers
78 views

Algorithms that connect neurons to previous layers as well as next

Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes ...
2
votes
0answers
50 views

Mini (local) NN for Mobile?

I'm working on architecture for a game AI where, due to the nature of the game, the classical approach seems likely be sufficient to beat most humans--the endgame is tractable and traditional game-...
2
votes
0answers
168 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
2
votes
0answers
138 views

Wide & Deep Learning Explanation

I was going through "Wide & Deep Learning" tensorflow tutorial & it's quite simply explained the process. But I missed few of the things. If someone can please explain them to me, it will be ...
2
votes
0answers
47 views

Recommendations on which architecture to use to guess appointment

I'm currently developping an application which allows psychologists to manage their schedule and budget. As a proof of concept, I would like to create an intelligent appointment service. There can be ...
2
votes
0answers
172 views

Is iLQG a good algorithm for model-based planning with simple environments?

In their work Continuous Deep Q-Learning with Model-based Acceleration, the author demonstrate great results of applying Imagination Rollouts for model-based acceleration of learning. They test their ...
2
votes
0answers
45 views

Loss of precision when encoding DNN weights

This question is related to the usage of NN in critical systems (those where a failure can cause life threatening situations - autopilots for example) and the need for formal guarantees on their ...
2
votes
0answers
208 views

Can anybody explain such behavior of accuracy and loss of my Net(caffe)?

I used this project for example(framework - caffe, arhitecture of net - mod of AlexNet, 400 images are used for training). I have this result: or this: Solver: ...
2
votes
0answers
119 views

Train, Validation and Test Split for Reporting Accuracy of Neural Model and BOW

I need to report accuracies of my neural model in a conference paper as compared to various baselines. What are the accepted standards for reporting accuracies in a fair manner? Neural Model: To be ...
2
votes
0answers
70 views

seq2seq vector to letters model

I'm looking to build a sequence-to-sequence model that takes in a 2048-long vector of 1s and 0s as my input and translating it to my known output of (a variable length) 1-20 long characters (ex. ...
2
votes
1answer
148 views

How do I classify an image that contains only polygons?

I have two closed polygons, drawn as connected straight black lines on a white background. I need to classify such images in to three forms Two separate polygons One polygon encloses the other The ...
2
votes
1answer
117 views

Why do the inputs and outputs of a convolutional layer usually have the same depth?

Here's the famous VGG-16 model. Do the inputs and outputs of a convolutional layer, before pooling, usually have the same depth? What's the reason for that? Is there a theory or paper trying to ...
2
votes
1answer
213 views

Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?

I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e....
1
vote
0answers
28 views

Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
1
vote
1answer
46 views

How to extract parameters from a text using AI/NLP

lets say I have three texts: "make a heading that says hello word" "make a heading of hello world" "create heading consist of hello world" How can I fetch those groups ...
1
vote
0answers
13 views

How can an “architectural motif” be extracted from a trained MLP?

I am trying to reproduce the paper Synthetic Petri Dish: A novel surrogate model for Rapid Architecture Search. In the paper, the authors try to reduce the architecture of an MLP model trained on ...
1
vote
0answers
17 views

For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?

When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations. For example, here (slide 20), given $o$ (the output, defined in slide 18), $z$ (the ...
1
vote
0answers
18 views

Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?

I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
1
vote
0answers
14 views

Drawing verticle lines on video frames in vehicle detection

I am working on an application of vehicle detection, the purpose is to check if road is congested or not. The solution that came to my mind is to detect the vehicles moving in each direction ...
1
vote
0answers
32 views

Designing Policy-Network for Deep-RL with Large, Variable Action Space

I am attempting a project involving training an agent to play a game using deep reinforcement learning. This project has a few features that complicate the design of the neural network: The action ...
1
vote
0answers
24 views

The last target name is missed in the test set

I am training a neural network with a dataset that has 51 classes and 6766 data in it. I used 80% for the training set, 10% for validation, and 10% for the test. After training I got confusion matrix ...
1
vote
0answers
57 views

Are Graph Neural Networks generalizations of Convolutional Neural Networks?

In lecture 4 of this course, the instructor argues that GNNs are generalizations of CNNs, and that one can recover CNNs from GNNs. He presents the following diagram (on the right) and mentions that it ...
1
vote
0answers
21 views

Papers on using symbolic methods as constraint on neural network?

Given a set of constraints on the input data, I am looking for papers that discuss using symbolic methods (decision trees, rule based, etc.) as a separate source of certainty in a classification task. ...
1
vote
0answers
21 views

Building a resume recommendation for a job post?

There are few challenges I am facing when building a resume recommendation for a particular job positing. Let's say we convert the resume into a vector on n-dimensions and job description also as an n-...
1
vote
0answers
58 views

Is the performance of a neural network, which was trained with encrypted data and weights, affected if the weights are decrypted?

Suppose that a neural network is trained with encrypted (for example, with homomorphic encryption and, more precisely, with the Paillier partial scheme) data. Moreover, suppose that it is also trained ...
1
vote
0answers
21 views

How do I infer exploding or vanishing gradients in Keras?

It may already be obvious that I am just a practitioner and just a beginner to Deep Learning. I am still figuring out lots of "WHY"s and "HOW"s of DL. So, for example, if I train a ...
1
vote
0answers
16 views

How does noise input size affect fake image generation with GANs?

In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. ...
1
vote
0answers
34 views

What is the gradient of an attention unit?

The paper Attention Is All You Need describes the Transformer architecture, which describes attention as a function of the queries $Q = x W^Q$, keys $K = x W^K$, and values $V = x W^V$: $\text{...
1
vote
0answers
15 views

Looking for a good approach for building an automated director for a racing game spectator mode

I'm building a tool that should assist a director to broadcast a racing game. I want this tool to suggest the human director which car to focus on and with which camera (among the available ones). I ...
1
vote
0answers
25 views

Why should variance(output) equal variance(input) in Xavier Initialisation?

In a lot of explanations online for Xavier Initialization, I see the following: With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a ...
1
vote
0answers
21 views

Neural network architecture with inputs and outputs being an unkown function each

I am trying to set up a neural network architecture that is able to learn the points of one function (blue curves) from the points of an other one (red curves). I think that it could be somehow ...
1
vote
0answers
16 views

React on train-validation curve after trening

I have a regression task that I tray to solve with AI. I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance) I understand basic ...
1
vote
0answers
52 views

How does NN follows law of energy conservation?

Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average ...
1
vote
0answers
24 views

How does Google's 2016 GNMT architecture work?

I'm trying to read this paper describing Google's LSTM architecture for machine translation from 2016. However, I'm getting stuck as certain things are described too vaguely for me. This is a picture ...
1
vote
0answers
34 views

How to use unmodified input in neural network?

My question may be a bit hard to explain... My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2. From this ...
1
vote
0answers
43 views

What are the use-cases of self-replicating neural networks?

I started researching the subject of self-replication in neural networks, and unexpectedly I saw that there is not much research on this subject. I should mention I am new in the field of NNs. This ...
1
vote
0answers
15 views

Effect of Batch Size on a Translation Model's Validation Scores

This is a general question. I am training a neural machine translation model with different batch sizes: 16, 32, and 64 for the same amount of epochs (10). I am getting very low validation scores for ...
1
vote
0answers
39 views

Where can I find pre-trained agents able to play games with multiple stages like exploration, dialog, combat?

My goal is to create an ML model to be able to classify different game stages, e.g., dialog with a non-player character, exploration, combat with enemy, in-game menu etc. In order to do that, I am ...
1
vote
0answers
33 views

In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?

While reading the Mutual Information Neural Estimation (MINE) paper [1] I came across section 3.2 Correcting the bias from the stochastic gradients. The proposed method requires the computation of the ...
1
vote
0answers
17 views

How does scheduled sampling for transformers work?

I was reading this paper which applies a modified version of the transformers for traffic forecasting. I am somewhat familiar with the transformer architecture and how it functions, but, in the paper, ...

1
4 5
6
7 8
14