Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

2
votes
1answer
22 views

How is equation 8 derived in the paper “Self-critical sequence training for image captioning”?

In the paper "Self-critical sequence training for image captioning", on page 3, they define the loss function (of the parameters $\theta$) of an image captioning system as the negative expected reward ...
2
votes
0answers
16 views

How to handle varying length of inputs that represent dependencies and recursivity in deep neural networks in case of regression?

I wanna solve a problem of regression to predict a factor. I decide to go with Deep Neural Networks as solution for my problem. The features in this problem represent loop characteristic such us loop ...
0
votes
1answer
19 views

Training a neural network to output the conditional probability of an event when the training data output is only binary

I have a dataset with hundreds of thousands of training examples. There are 27 input variables and one output variable which is always a 0 or a 1, based on whether an event happened or not. My ...
1
vote
2answers
24 views

Relative Importance of Input Features

I am confused as to how neural networks consider the different features by that have access to at the input layer. Consider this example: I have three features: an image, a dollar amount, and a ...
0
votes
1answer
18 views

Loss function spikes

For the UNSW-NB15 dataset i receive spikes in the loss function during training. The algorithms see part of this UNSW dataset a single time. Loss function is plotted after every batch. For other ...
3
votes
0answers
19 views

Symbolic “math” using trained networks

Does anyone work out ways of relating trained networks by symbolic logic? For example: If I train a network on pictures of dogs, and I train a network on pictures of shirts. You could imagine that ...
1
vote
2answers
31 views

Modelling gut-feeling/subconscious knowledge of stock market traders

Some (stock market) traders have the ability to produce a high percentage of winning trades (80%+, positive return) over years. I had the chance to look into real money trades of two such traders and ...
4
votes
1answer
46 views

Is fuzzy logic connected to neural networks?

Fuzzy logic is typically used in control theory and engineering applications, but is it connected fundamentally to classification systems? Once I have a trained neural network (multiple inputs, one ...
1
vote
2answers
34 views

To what does the number of hidden layers in a neural network correspond?

In a neural network, the number of neurons in the hidden layer corresponds to the complexity of the model generated to map the inputs to output(s). More neurons creates a more complex function (and ...
1
vote
2answers
66 views

Find object location (x, y) in an image

I am generating images that consist of points where the object's location is where the most overlap of points occurs. In this example, the object location is (25, 51). I am trying to train a model to ...
1
vote
1answer
43 views

How does backpropagation with unbounded activation functions such as ReLU work?

I am in the process of writing my own basic machine learning library in Python as an exercise to gain a good conceptual understanding. I have successfully implemented backpropagation for activation ...
0
votes
0answers
50 views

Using AI to guess a mathematical pattern of certain polynomials in four variables: practical challenge

I'd like to use machine learning to guess a mathematical pattern: the input are certain polynomials in four variables $q_1,q_2,q_3,q_4$, the output can be zero or one. Allowed polynomials are such ...
0
votes
0answers
16 views

How can I train a neural network to predict what cars are going to be liked from an auction? PYTHON

so I'm trying to make a program where I can predict what cars from an auction are going to be liked by a specific client based on the previous cars he has bought. I am coding this program in python. ...
0
votes
0answers
13 views

DCGAN generator accuracy doesnt improve for high-res images

I trained a DCGAN on MNIST and CelebA dataset with 28x28 image size. Both the models were able to train successfully. I used many tips from https://github.com/soumith/ganhacks to make both the G and D'...
1
vote
0answers
14 views

Feature visualization on neural networks which are not for classification

Feature visualization allows to better understand neural networks by generating images that maximize the activation of a specific neuron, and therefore understand what are the abstract features that ...
0
votes
1answer
87 views

What is the most biologically plausible representation for the actor and critic?

Which representation is most biologically plausible for actor nodes? For example, actions represented across several output nodes which may be either mutually exclusive with each other (e.g., go ...
0
votes
1answer
26 views

Choosing an instance in AWS or GCP for training a CNN

I'm new to the whole world of AI and neural networks, and I'm looking to get started with training a CNN on a cloud service like AWS or GCP but I'm totally overwhelmed by the choices for VM instances. ...
1
vote
1answer
33 views

Is it a great misconception that the softmax is an activation function?

An activation function is a function from $R \rightarrow R$. It takes as input the inner products of weights and activations in the previous layer. It outputs the activation. A softmax however, is a ...
0
votes
1answer
12 views

Should you reload the optimizer for transfer learning?

For example, you train on dataset 1 with an adaptive optimizer like Adam. Should you reload the learning schedule etc from the end of training on dataset1 when attempting transfer to dataset2? Why or ...
1
vote
0answers
30 views

Can traditional neural networks be combined with spiking neural networks?

Can traditional neural networks be combined with spiking neural networks? And can there be training algorithms for such hybrid network? Does such hybrid network model biological brains? As I ...
0
votes
0answers
24 views

Can a learning rate graph look unusual and weird?

I am trying to model a simple Neural Net to classify data amongst 14 classes. The data is quite high dimensional, with 21392 rows and 1970 columns, with the last column being the labels (which have ...
1
vote
1answer
54 views

A neural network for digits recognition doesn't work (MNIST, Numpy) [closed]

I'm a beginner in machine learning and I was trying to make a test neural network for digits recognition from scratch using Numpy. I used MNIST dataset for training and testing. Input layer have 28*28 ...
1
vote
1answer
70 views

How do I convert table-based to neural network-based Q-learning?

I've used a table to represent the Q function, while an agent is being trained to catch the cheese without touching the walls. The first and last row (and column) of the matrix are associated with ...
0
votes
0answers
33 views

What are the practical and theoretical properties that commonly used loss functions have?

What are the practical and theoretical properties that commonly used loss functions have (in particular, in the context of neural networks)?
3
votes
4answers
85 views

Choose model with the smallest loss or highest accuracy?

I have two Neural Network models (I use LSTM) that have different result on validation set (~100 samples data): Model A: Accuracy: ~91%, Loss: ~0.01 Model B: Accuracy: ~83%, Loss: ~0.003 The size ...
2
votes
0answers
26 views

Using neuroevolution for something other than games

I'm seeing a lot of examples involving games, or robot problems. What about how to make something else conform to this framework? How do you transform say a csv file of psychological data to determine ...
2
votes
0answers
11 views

Batch PTA stopping condition

I am reviewing my Neural Network lectures and I have a doubt: My book's (Haykin) batch PTA describes a cost function which is defined over the set of the misclassified inputs. I have always been ...
2
votes
1answer
16 views

What are the aspects that most impact on the inference time for neural networks in embedded systems?

I work with neural networks for real-time image processing on embedded softwares and I tested different architectures (Googlenet, Mobilenet, Resnet, custom networks...) and different hardware ...
0
votes
0answers
19 views

Models at RunTime and Artificial Intelligence

I'm looking for some help... Someone can refer me or recommend recent studies that relate to the models at runtime with the AI, that would really help me a lot. I also appreciate your comments on the ...
2
votes
0answers
30 views

Which Neuron represent which part of the non-linear feature?

In any neural network, each neuron in the network represents some part of non-linear feature of the input. Ex: Like in mnist data, Consider the stem of number 9 is cut into multiple pieces and ...
1
vote
0answers
12 views

Sample from a distribution inside a NN layer

Is it possible to sample from a distribution inside a neural network forward function? Assume that there is a NN and a sample is needed to be derived from it at every forward-pass to randomly set a ...
1
vote
0answers
31 views

How to create meaningful multiple object detection evaluation comparison graph?

I have got multi-class object detector. One model accuracy evaluation of detection consists of: mAP, FP, FN, TP for each class divided to two graphs and looks like this (I've used this repo for ...
0
votes
1answer
42 views

Reduce receptive field size of CNN while keeping its capacity?

I have a convolutional encoder (a CNN) consisting of DenseBlocks and a total of 50 layers (cf. FC-DenseNet103). The receptive field of the encoder (after last layer) is 660 according to Tensorflow ...
0
votes
1answer
48 views

Which field to study to learn & create a.i generated simulations?

I wasn't sure how to title this question so pardon me please. You may have seen at least one video of those "INSANE A.I created simulation of {X} doing {Y & Z} like the following ones: A.I ...
3
votes
1answer
33 views

How sigmoid funtion helps us in reducing error in neural networks?

As in sigmoid function when input x is very large or very small the curve is flat that means low gradient descent but when it is in between the slope is more so, My question is how this thing helps us ...
1
vote
0answers
57 views

An idea I had to create the first Humanoid using Deep Learning

I've come up with an idea on how we could use a combination of Deep Learning and body sensors to create a walking talking living humanoid. Here goes: First, we will recruit 1 billion people and have ...
2
votes
1answer
63 views

Can TD-Lambda be used with Deep Reinforcement Learning?

TD lambda is a way to interpolate between TD(0) - bootstraping over a single step, and, TD(max), bootstraping over the entire episode length, or, Monte carlo. Reading the link above, I see that an ...
0
votes
1answer
24 views

What role the activation function plays in the forward pass and how it is different from backpropagation

Is the role played by activation function significant only during the training of neural network or they play their role during testing (after training we supply data for prediction) the network. I ...
0
votes
1answer
34 views

Tips for keeping the distribution of weights normal

I work on a project where the neural network must be quantized on 8 or 16 bits for an embedded platform, thus I lose some precision. EDIT: since our platform does not have floating point arithmetic ...
0
votes
0answers
13 views

How to shape the weights or nodes during gradient training of neural network? Training with constraints?

Gradient training changes indiscriminately all the weights and nodes of the neural network. But one can imagine the situations when the training should be shaped, e.g.: One can put constraints on ...
2
votes
1answer
28 views

Will LMS always be convex function? If yes, then why do we change it for neural networks?

In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least ...
1
vote
2answers
62 views

Choosing the right neural network settings

I'm trying to train a neural network on evaluating chess positions if rather white (0.0) or black would win (1.0) Currently the input consists of 4 bits per chess field (piece id 0 - 12, equals 64*4)....
-1
votes
0answers
22 views

How many gradient descent iterations does it take for a neural network to over-fit, relative to network “complexity”

Following this question, I have a more refined, general, and probably more answerable question: Assuming: Infinite training data A learning problem whose loss function has no local minima (for ...
1
vote
1answer
23 views

Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?

I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure: ...
3
votes
2answers
70 views

Are neural networks statistical models?

By reading the abstract of Neural Networks and Statistical Models paper it would seem that ANNs are statistical models. In contrast Machine Learning is not just glorified Statistics. I am looking ...
-1
votes
0answers
20 views

Pre-trained regression models with high precission?

I'm trying to solve a regression problem with neural nets where my input is a set of 3 real numbers and my output is a set of 6 real numbers. I can generate as much sets as I want, so amount of data ...
1
vote
1answer
50 views

Is there a relation between the size of the neural networks and speed of convergence in deep reinforcement learning?

Is there a connection between the approximator network sizes in a RL task and the speed of convergence to an (near) optimal policy or value function? When thinking about this, I came across the ...
1
vote
1answer
43 views

Is my Neural Network program fully connected?

I have the following program for my neural network: ...
3
votes
1answer
30 views

is it possible to find same persons based on text analyzing?

i needed to make a system for recognizing people based on hundreds of text by finding similarities in their written text grammatically or similarities between words they choose for writing , i don't ...
0
votes
1answer
17 views

NEAT Algorithm,Does Innovation weights shared?

I am trying to implement Neat Algorithm for python from scratch. However I am stucked. When a new innovation number created it has two nodes which represents the connection. Also this innovation ...