Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

584 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
2answers
231 views

Back-of-the-envelope machine learning (specifically neural networks) calculations

There is a popular story regarding the back-of-the-envelope calculation performed by a British physicist named G. I. Taylor. He used dimensional analysis to estimate the power released by the ...
5
votes
0answers
29 views

What is the impact of using multiple BMUs for self-organizing maps?

Sort of a conceptual question here. I was implementing a SOM algorithm to better understand its variations and parameters and got curious about one bit: the BMU (best matching unit == the neuron that ...
5
votes
1answer
858 views

What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?

I was following some examples to get familiar with TensorFlow's LSTM API, but noticed that all LSTM initialization functions require only the num_units parameter, ...
5
votes
2answers
154 views

Why do layered neural nets struggle with continous data?

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. It was my understanding that this was the ...
5
votes
1answer
112 views

Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?

I struggle to find Rosenblatt's perceptron training algorithm in any of his publications from 1957 - 1961, namely: Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms The ...
5
votes
1answer
178 views

For each epoch, can I use only on a subset of the full training dataset to train the neural network?

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I ...
5
votes
0answers
133 views

How to design 4D Deep Recurrent Neural Networks using Tensorflow?

I want to design a simple model that predicts the movement of coordinates with RNNs. In a typical three-dimensional LSTM model, one feature is encoded as one hot encoding, and the ...
5
votes
1answer
53 views

Which marketing-related classification challenges is a feed forward neural network suited to solve?

I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. Any ideas?
4
votes
0answers
75 views

Forcing a neural network to be close to a previous model - Regularization through given model

I'm wondering, has anyone seen any paper where one trains a network but biases it to produce similar outputs to a given model (such as one given from expert opinion or it being a previously trained ...
4
votes
0answers
62 views

What should I do with the flatten layer during back-propagation?

I'm creating a CNN network without other frameworks such as PyTorch, Keras, Tensorflow, and so on. During the forward pass, the Flatten layer reshapes the previous ...
4
votes
0answers
21 views

How do weights changes handles during back-propagation when there are unknown labels

I have a question about how weights are updated during back-propagation for some of my samples that have unknown labels (please note, unknown, not missing). The reason they are unknown is because this ...
4
votes
0answers
43 views

Is it a good idea to first train a spiking neural network and then convert it to a conventional neural network?

In many papers about artificial spiking neural networks (SNNs), the performance of them is not up to par with traditional ANNs. I have read how some people have converted ANNs to SNNs using various ...
4
votes
0answers
71 views

Is there a mathematical formula that describes the learning curve in neural networks?

In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal. Is there a mathematical ...
4
votes
0answers
77 views

Are there datasets to solve differential equations in a supervised fashion?

Are there datasets to solve differential equations in a supervised fashion? More precisely, the input is a differential equation and the label should be the general solution to that differential ...
4
votes
1answer
67 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
4
votes
0answers
139 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
4
votes
0answers
51 views

Is it possible to control asymptotic behaviour of neural network models?

Is it possible to specify what the asymptotic behaviour of a Neural Networks (NN) model should be? I am thinking on NN which try to learn a mapping $\vec y=f(\vec x)$ with $\vec x$ a vector of ...
4
votes
0answers
70 views

Why isn't the evolutionary Turing machine mainstream?

Given that recurrent neural networks are equivalent to a Turing machine, then why isn't the evolutionary Turing machine, e.g. described in the paper Evolution of evolution: Self-constructing ...
4
votes
0answers
26 views

What are some neural network models that can use auxiliary info during training for image segmentation?

What are some deep learning models that can use supplementary information other than RGB channels for image segmentation? For example imagine a poorly shot image of a river (blue) that shows a gap, ...
4
votes
0answers
238 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
4
votes
1answer
103 views

How do big companies, like Facebook, model individuals and their interaction?

As a layman in AI, I want to get an idea of how big data players, like Facebook, model individuals (of which they have so many data). There are two scenarios I can imagine: Neural networks build ...
4
votes
0answers
60 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
4
votes
0answers
41 views

Can there be applications of byzantine neural networks on quantum computers?

This question came after I connected 2 pieces of information : I recently listened to The Byzantine Generals’ Problem, Poisoning, and Distributed Machine Learning with El Mahdi El Mhamdi (Beneficial ...
4
votes
2answers
2k views

What is the purpose of “reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval.”?

I am a deep learning beginner recently reading this book "Deep learning with Python", the example explains the process of implementing a greyscale image classification using MNIST in keras, in the ...
4
votes
0answers
2k views

Is this a good way to represent Connect 4 to a Neural Network?

I'm attempting to make a bot for the Connect 4 competition on http://riddles.io My bot isn't horrible, like it's getting up the ladder, but it cannot compete with the winning bots. I'm using a ...
4
votes
1answer
57 views

GPU/TPU acceleration for neural networks with various network topologies

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
3
votes
0answers
41 views

How does backpropagation work in LSTMs?

After reading a lot of articles (for instance, this one Understanding LSTM Networks), I know that the long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in ...
3
votes
1answer
34 views

What should the dimension of the input be for text summarization?

I am trying to build a model for extractive text summarization using keras sequential layers. I am having a hard time trying to understand how to input my x data. Should it be an array of documents ...
3
votes
1answer
97 views

What are the differences between artificial neural networks and other function approximators?

Modern artificial neural networks use a lot more functions than just the classic sigmoid, to the point I'm having a hard time really seeing what classifies something as a "neural network" over other ...
3
votes
0answers
50 views

Understanding the TensorFlow implementation of the policy gradient method

I was trying to understand the implementation of a basic policy gradient (REINFORCE) method using TensorFlow. I think I got almost everything. The only thing that still bothers me is the loss function ...
3
votes
0answers
135 views

Understanding the results of “Visualizing and Understanding Convolutional Networks”

I am trying to understand the results of the paper Visualizing and Understanding Convolutional Networks, in particular the following image: What are these 3x3 blocks and their 9 cells representing? ...
3
votes
0answers
55 views

How to implement Mean square error loss function in mini batch GD

I have a vectorized implementation of the neural network in c++. I successfully solve the classification problems of Fashion MNIST and CIFAR. Now I am modifying my code to do the Linear regression. I ...
3
votes
0answers
24 views

Is there any way of generating fixed-length sequences with RNNs?

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
3
votes
0answers
79 views

Are there principled ways of tuning a neural network in case of overfitting and underfitting?

Whenever I tune my neural network, I usually take the common approach of defining some layers with some neurons. If it overfits, I reduce the layers, neurons, add dropout, utilize regularisation. ...
3
votes
1answer
63 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
3
votes
1answer
50 views

How to solve the problem of variable-sized AST as input for a (convolutional) neural network model?

In my work I have a given source code for a module. From this module I generate an AST, whose size is dependent on the size of the module (e.g. more source code -> bigger AST). I want to train a ...
3
votes
0answers
39 views

What are some ways to quickly evaluate the potential of a given NN architecture?

Main question Is there some way we can leverage general knowledge of how certain hyperparameters affect performance, to very rapidly get some sort of estimate for how good a given architecture could ...
3
votes
0answers
31 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
3
votes
1answer
90 views

Is running more epochs really a direct cause of overfitting?

I've seen some comments in online articles/tutorials or Stack Overflow questions which suggest that increasing number of epochs can result in overfitting. But my intuition tells me that there should ...
3
votes
0answers
45 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
3
votes
1answer
57 views

How to classify human actions?

I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses). I want to classify human actions real-time like: Left-arm bended Arm above ...
3
votes
1answer
56 views

What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?

I try to solve some easy functions with a neuronal network (aforge-lib): This is how I generate the dataset: ...
3
votes
0answers
35 views

Reinforcement Learning on quantum circuit

I am trying to teach an agent to make any random 1-qubit state reach uniform superposition. So basically, the full circuit will be ...
3
votes
4answers
93 views

Regression using neural network

I'd like to ask for any kind of assistance regarding the following problem: I was given the following training data: 100 numbers, each one is a parameter, they together define a number X(also given)....
3
votes
0answers
85 views

Why do regression LSTMs learn high to low inputs significantly better than low to high?

The specific problem I have is learning the relation $x^2$. I have an array of 0 through 19 (input values) and a target array of 0, 1, 4, 9, 16, 25, 36 and so on all the way up to $19^2$=361. I have ...
3
votes
0answers
73 views

Deepfakes as “force for good”?

As per the law of unintended consequences, could it be that deepfakes will eventually have the opposite effect to what people currently seem to fear most. For example, once it is clear that anyone can ...
3
votes
0answers
50 views

Ideas on a network that can translate image differences into motor commands?

I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image ...
3
votes
1answer
68 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
3
votes
0answers
19 views

Which hyper-parameters are considered in neural architecture search?

I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters ...
3
votes
1answer
75 views

When should I create a custom loss function?

I'm using a neural network to solve a multi regression problem because I'm trying to predict continuous values. To be more specific, I'm making a tracking algorithm to track the position of an object, ...

1
2 3 4 5
12