Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

468 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
10
votes
1answer
284 views

Can layers of deep neural networks be seen as Hopfield networks?

Hopfield networks are able to store a vector and retrieve it starting from a noisy version of it. They do so setting weights in order to minimize the energy function when all neurons are set equal to ...
7
votes
1answer
717 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
6
votes
1answer
300 views

How come that the addition of features can decrease the performance of a neural network?

I have a Remaining Useful Life (RUL) prediction problem that I want to solve. When I added two or more features as inputs to my ANN, the accuracy of my ANN has been decreased. More precisely, I've ...
5
votes
0answers
77 views

Learning from NNs

When a neural network learns something from a data set, we are left with a bunch of weights which represent some approximation of knowledge about the world. Although different data sets or even ...
5
votes
0answers
89 views

How can we prove that an autoassociator network will continue to perform if we zero the diagonal elements of a weight matrix?

How can we prove that an auto-associator network will continue to perform if we zero the diagonal elements of a weight matrix that has been determined by the Hebb rule? In other words, suppose that ...
5
votes
0answers
27 views

What is the impact of using multiple BMUs for self-organizing maps?

Sort of a conceptual question here. I was implementing a SOM algorithm to better understand its variations and parameters and got curious about one bit: the BMU (best matching unit == the neuron that ...
5
votes
2answers
139 views

Why do layered neural nets struggle with continous data?

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. It was my understanding that this was the ...
5
votes
1answer
85 views

Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?

I struggle to find Rosenblatt's perceptron training algorithm in any of his publications from 1957 - 1961, namely: Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms The ...
5
votes
1answer
196 views

Back-of-the-envelope machine learning (specifically neural networks) calculations

There is a popular story regarding the back-of-the-envelope calculation performed by a British physicist named G. I. Taylor. He used dimensional analysis to estimate the power released by the ...
5
votes
0answers
127 views

How to design 4D Deep Recurrent Neural Networks using Tensorflow?

I want to design a simple model that predicts the movement of coordinates with RNNs. In a typical three-dimensional LSTM model, one feature is encoded as one hot encoding, and the ...
5
votes
1answer
52 views

Which marketing-related classification challenges is a feed forward neural network suited to solve?

I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. Any ideas?
4
votes
0answers
13 views

How do weights changes handles during back-propagation when there are unknown labels

I have a question about how weights are updated during back-propagation for some of my samples that have unknown labels (please note, unknown, not missing). The reason they are unknown is because this ...
4
votes
0answers
33 views

Is it a good idea to first train a spiking neural network and then convert it to a conventional neural network?

In many papers about artificial spiking neural networks (SNNs), the performance of them is not up to par with traditional ANNs. I have read how some people have converted ANNs to SNNs using various ...
4
votes
0answers
58 views

Is there a mathematical formula that describes the learning curve in neural networks?

In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal. Is there a mathematical ...
4
votes
1answer
391 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
4
votes
1answer
59 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
4
votes
0answers
67 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
4
votes
0answers
42 views

Is it possible to control asymptotic behaviour of neural network models?

Is it possible to specify what the asymptotic behaviour of a Neural Networks (NN) model should be? I am thinking on NN which try to learn a mapping $\vec y=f(\vec x)$ with $\vec x$ a vector of ...
4
votes
0answers
69 views

Why isn't the evolutionary Turing machine mainstream?

Given that recurrent neural networks are equivalent to a Turing machine, then why isn't the evolutionary Turing machine, e.g. described in the paper Evolution of evolution: Self-constructing ...
4
votes
0answers
190 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
4
votes
1answer
77 views

Psychological models at Facebook et al

As a layman in AI I want to get an idea how big data players like Facebook model individuals (of which they have so many data). There are two scenarios I can imagine: Neural networks build clusters ...
4
votes
0answers
45 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
4
votes
0answers
39 views

Can there be applications of byzantine neural networks on quantum computers?

This question came after I connected 2 pieces of information : I recently listened to The Byzantine Generals’ Problem, Poisoning, and Distributed Machine Learning with El Mahdi El Mhamdi (Beneficial ...
4
votes
2answers
504 views

Handling emotion in informal text (Hi vs HIIIIII!!!!)?

This is a question related to Neural network to detect "spam"?. I'm wondering how it would be possible to handle the emotion conveyed in text. In informal writing, especially among a ...
4
votes
2answers
1k views

What is the purpose of “reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval.”?

I am a deep learning beginner recently reading this book "Deep learning with Python", the example explains the process of implementing a greyscale image classification using MNIST in keras, in the ...
4
votes
1answer
791 views

How do you handle multiple categorical values in a single column for wide_deep model in tensorflow?

To start, let me just say that I am very new to tensorflow and Machine Learning in general. But, as part of my learning project I am trying to adapt the tensorflow wide and deep model to generate ...
4
votes
0answers
2k views

Is this a good way to represent Connect 4 to a Neural Network?

I'm attempting to make a bot for the Connect 4 competition on http://riddles.io My bot isn't horrible, like it's getting up the ladder, but it cannot compete with the winning bots. I'm using a ...
3
votes
0answers
43 views

Are there examples of neural networks (used for control) implemented on a FPGA or on a neurochip?

Greetings to all respected colleagues! I want to consult on the use of FPGAs and neurochips. I plan to use it in my laboratory project for programming control systems on neural networks. In my work,...
3
votes
0answers
17 views

Is there any way of generating fixed-length sequences with RNNs?

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
3
votes
0answers
70 views

Are there principled ways of tuning a neural network in case of overfitting and underfitting?

Whenever I tune my neural network, I usually take the common approach of defining some layers with some neurons. If it overfits, I reduce the layers, neurons, add dropout, utilize regularisation. ...
3
votes
1answer
45 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
3
votes
1answer
40 views

How to solve the problem of variable-sized AST as input for a (convolutional) neural network model?

In my work I have a given source code for a module. From this module I generate an AST, whose size is dependent on the size of the module (e.g. more source code -> bigger AST). I want to train a ...
3
votes
0answers
36 views

What are some ways to quickly evaluate the potential of a given NN architecture?

Main question Is there some way we can leverage general knowledge of how certain hyperparameters affect performance, to very rapidly get some sort of estimate for how good a given architecture could ...
3
votes
0answers
25 views

Rarely predict minority class imbalanced datasets

I have a dataset in which class A has 99.8%, class B 0.1% and class C 0.1%. If I train my model on this dataset, it predicts always class A. If I do oversampling, it predicts the classes evenly. I ...
3
votes
0answers
58 views

Is running more epochs really a direct cause of overfitting?

I've seen some comments in online articles/tutorials or Stack Overflow questions which suggest that increasing number of epochs can result in overfitting. But my intuition tells me that there should ...
3
votes
0answers
41 views

How does the memory mechanism (reading and writing) work in a neural Turing machine?

In neural Turing machine (NTM), reading memory is represented as \begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align} and writing to memory is represented as ...
3
votes
1answer
46 views

How to classify human actions?

I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses). I want to classify human actions real-time like: Left-arm bended Arm above ...
3
votes
1answer
45 views

What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?

I try to solve some easy functions with a neuronal network (aforge-lib): This is how I generate the dataset: ...
3
votes
0answers
30 views

Reinforcement Learning on quantum circuit

I am trying to teach an agent to make any random 1-qubit state reach uniform superposition. So basically, the full circuit will be ...
3
votes
0answers
81 views

Why do regression LSTMs learn high to low inputs significantly better than low to high?

The specific problem I have is learning the relation $x^2$. I have an array of 0 through 19 (input values) and a target array of 0, 1, 4, 9, 16, 25, 36 and so on all the way up to $19^2$=361. I have ...
3
votes
0answers
48 views

Does MMD-VAE solve the problem of blurred images of vanilla VAEs?

I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the ...
3
votes
0answers
60 views

Deepfakes as “force for good”?

As per the law of unintended consequences, could it be that deepfakes will eventually have the opposite effect to what people currently seem to fear most. For example, once it is clear that anyone can ...
3
votes
0answers
44 views

Ideas on a network that can translate image differences into motor commands?

I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image ...
3
votes
1answer
47 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
3
votes
0answers
17 views

Which hyper-parameters are considered in neural architecture search?

I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters ...
3
votes
1answer
53 views

when should I create a custom loss function?

Hi I'm using neural network to solve a multi regression problem. I'm trying to predict continuous values, to be more specific I'm making a tracking algorithm to track the position of an Object, I'm ...
3
votes
0answers
57 views

Are there datasets to solve differential equations in a supervised fashion?

Are there datasets to solve differential equations in a supervised fashion? More precisely, the input is a differential equation and the label should be the general solution to that differential ...
3
votes
0answers
39 views

Vector normalization by a neural network

I'm wondering if there is a NN that can achieve the following task: Output a unit vector that is parallel to the input vector. i.e., input a vector $\mathbf{v}\in\mathbb{R}^d$, output $\mathbf{v}/\|\...
3
votes
1answer
60 views

What sort of Neural Network is best suited to predicting a future purchase?

I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4. Now I'm trying to do a NN that can make a prediction. ...
3
votes
0answers
21 views

Handwritten digits recognition during the process of writing

I know how to train a NN for recognizing handwritten digits (e.g. using the MNIST database). I'm wondering how to accomplish the same "online", which is during the process of writing e.g. I'm writing ...

1
2 3 4 5
10