Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

792 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
12
votes
0answers
421 views

How can Viv generate new code based on some user's query?

I have been looking into Viv, an artificial intelligent agent in development. Here is a demonstration of Viv (by Dag Kittlaus). Based on what I understand, this AI can generate new code and execute it ...
11
votes
2answers
203 views

Why does Batch Normalization work?

Adding BatchNorm layers improves training time and makes the whole deep model more stable. That's an experimental fact that is widely used in machine learning practice. My question is - why does it ...
9
votes
0answers
141 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
8
votes
2answers
319 views

Back-of-the-envelope machine learning (specifically neural networks) calculations

There is a popular story regarding the back-of-the-envelope calculation performed by a British physicist named G. I. Taylor. He used dimensional analysis to estimate the power released by the ...
7
votes
0answers
74 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
6
votes
1answer
54 views

What is the impact of using multiple BMUs for self-organizing maps?

Here's a sort of a conceptual question. I was implementing a SOM algorithm to better understand its variations and parameters. I got curious about one bit: the BMU (best matching unit == the neuron ...
6
votes
0answers
87 views

How do neural network topologies affect GPU/TPU acceleration?

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
6
votes
1answer
89 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
6
votes
1answer
189 views

Why do layered neural nets struggle with continous data?

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. It was my understanding that this was the ...
5
votes
1answer
145 views

How do big companies, like Facebook, model individuals and their interaction?

As a layman in AI, I want to get an idea of how big data players, like Facebook, model individuals (of which they have so many data). There are two scenarios I can imagine: Neural networks build ...
5
votes
0answers
166 views

How to design 4D Deep Recurrent Neural Networks using Tensorflow?

I want to design a simple model that predicts the movement of coordinates with RNNs. In a typical three-dimensional LSTM model, one feature is encoded as one hot encoding, and the ...
5
votes
1answer
55 views

Which marketing-related classification challenges is a feed forward neural network suited to solve?

I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. Any ideas?
4
votes
0answers
41 views

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems? If we take, for example, the travelling salesman problem (or the dominating set problem). Let's say ...
4
votes
0answers
48 views

Can the quality of randomness in neural network initialization affect model fitting?

This is a topic I have been arguing about for some time now with my colleagues, maybe you could also voice your opinion about it. Artificial neural networks use random weight initialization within a ...
4
votes
0answers
49 views

Visualizing the Loss Landscape of Neural Nets: Meaning of the word 'filter'?

I found myself scratching my head when I read the following phrase in the paper Visualizing the Loss Landscape of Neural Nets: To remove this scaling effect, we plot loss functions using filter-wise ...
4
votes
0answers
58 views

Why are most commonly used activation functions continuous?

I have come to notice that the most commonly used activation functions are continuous. Is there any specific reason behind this? Results such as this paper have worked on training networks with ...
4
votes
0answers
67 views

When do two identical neural networks have uncorrelated errors?

In Chapter 9, section 9.1.6, Raul Rojas describes how committees of networks can reduce the prediction error by training N identical neural networks and averaging the results. If $f_i$ are the ...
4
votes
0answers
45 views

When is using weight regularization bad?

Regularization of weights (e.g. L1 or L2) keeps them small and standardized, which can help reduce data overfitting. From this article, regularization sounds favorable in many cases, but is it always ...
4
votes
0answers
75 views

Has the logistic map ever been used as an activation function?

I find the logistic map absolutely fascinating. Both in itself (because I love fractal) and because it is observed in nature (see: https://www.youtube.com/watch?v=ovJcsL7vyrk). I'm wondering if anyone ...
4
votes
3answers
132 views

Why neural networks tend to be trained to recognize multiple things instead of just one?

I was watching this series: https://www.youtube.com/watch?v=aircAruvnKk The series demonstrates neural networks by building a simple number recognizing network. It got me thinking: Why neural networks ...
4
votes
0answers
82 views

Forcing a neural network to be close to a previous model - Regularization through given model

I'm wondering, has anyone seen any paper where one trains a network but biases it to produce similar outputs to a given model (such as one given from expert opinion or it being a previously trained ...
4
votes
0answers
28 views

How do weights changes handles during back-propagation when there are unknown labels

I have a question about how weights are updated during back-propagation for some of my samples that have unknown labels (please note, unknown, not missing). The reason they are unknown is because this ...
4
votes
0answers
50 views

Is it a good idea to first train a spiking neural network and then convert it to a conventional neural network?

In many papers about artificial spiking neural networks (SNNs), the performance of them is not up to par with traditional ANNs. I have read how some people have converted ANNs to SNNs using various ...
4
votes
0answers
90 views

Is there a mathematical formula that describes the learning curve in neural networks?

In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal. Is there a mathematical ...
4
votes
0answers
301 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
4
votes
1answer
335 views

What are the differences between Bytenet and Wavenet?

I recently read Bytenet and Wavenet and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the ...
4
votes
0answers
86 views

Why isn't the evolutionary Turing machine mainstream?

Given that recurrent neural networks are equivalent to a Turing machine, then why isn't the evolutionary Turing machine, e.g. described in the paper Evolution of evolution: Self-constructing ...
4
votes
0answers
36 views

What are some neural network models that can use auxiliary info during training for image segmentation?

What are some deep learning models that can use supplementary information other than RGB channels for image segmentation? For example imagine a poorly shot image of a river (blue) that shows a gap, ...
4
votes
0answers
374 views

What is the difference between GAT and GaAN?

I was looking at two papers Graph Attention Networks (GAT) by Petar Veličković and GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang. I'm trying to ...
4
votes
0answers
83 views

How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?

It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on ...
4
votes
0answers
41 views

Can there be applications of byzantine neural networks on quantum computers?

This question came after I connected 2 pieces of information : I recently listened to The Byzantine Generals’ Problem, Poisoning, and Distributed Machine Learning with El Mahdi El Mhamdi (Beneficial ...
4
votes
0answers
113 views

What characteristics make it difficult for a Neural Network to approximate a function?

What are the characteristics which make a function difficult for the Neural Network to approximate? Intuitively, one might think uneven functions might be difficult to approximate, but uneven ...
4
votes
0answers
2k views

Is this a good way to represent Connect 4 to a Neural Network?

I'm attempting to make a bot for the Connect 4 competition on http://riddles.io My bot isn't horrible, like it's getting up the ladder, but it cannot compete with the winning bots. I'm using a ...
3
votes
1answer
95 views

Which ANN structure to use?

Let $\mathcal{S}$ be the training input data set where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design a ANN so that the cost function below is minimized (the sum of square of ...
3
votes
0answers
41 views

Why does sigmoid saturation prevent signal flow through the neuron?

As per these slides on page 35: Sigmoids saturate and kill gradients. when the neuron's activation saturates at either tail of 0 or 1, the gradient at these regions is almost zero. the gradient and ...
3
votes
0answers
43 views

Is non-negative matrix factorization for machine learning obsolete?

I am taking a course about using matrix factorization for machine learning. The first thing that came into my mind is by using the matrix factorization we are always limited to linear relationships ...
3
votes
1answer
107 views

What is asymmetric relaxation backpropagation?

In Chapter 8, section 8.5.2, Raul Rojas describes how the weights for a layer of a neural network can be calculated using a pseudoinverse of the sigmoid function in the nodes, he explains this is an ...
3
votes
0answers
59 views

Why is there a Uniform and Normal version of He / Xavier initialization in DL libraries?

Two of the most popular initialization schemes for neural network weights today are Xavier and He. Both methods propose random weight initialization with a variance dependent on the number of input ...
3
votes
0answers
34 views

Image classification - Need method to classify “unknown” objects as “trash” (3D objects)

We have an image classifier that was built using CNN with faster R-CNN and Yolov5. It is designated to run on 3D objects. All of those objects have similar "features" structure, but the ...
3
votes
0answers
97 views

Doing backpropagation in an Tensorflow.js Neural Network

I have a neural network (which I am making from scratch). In order to make the neural network "learn" I need to conduct back-propagation. Using the code at the below how would I conduct back-...
3
votes
1answer
49 views

How can I create an embedding layer to convert words to a vector space from scratch?

For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some ...
3
votes
0answers
45 views

How does backpropagation work in LSTMs?

After reading a lot of articles (for instance, this one Understanding LSTM Networks), I know that the long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in ...
3
votes
0answers
73 views

Understanding the TensorFlow implementation of the policy gradient method

I was trying to understand the implementation of a basic policy gradient (REINFORCE) method using TensorFlow. I think I got almost everything. The only thing that still bothers me is the loss function ...
3
votes
0answers
154 views

Understanding the results of “Visualizing and Understanding Convolutional Networks”

I am trying to understand the results of the paper Visualizing and Understanding Convolutional Networks, in particular the following image: What are these 3x3 blocks and their 9 cells representing? ...
3
votes
0answers
217 views

How to implement Mean square error loss function in mini batch GD

I have a vectorized implementation of the neural network in c++. I successfully solve the classification problems of Fashion MNIST and CIFAR. Now I am modifying my code to do the Linear regression. I ...
3
votes
0answers
31 views

Is there any way of generating fixed-length sequences with RNNs?

Is there any way of generating fixed-length sequences with RNNs? I want to tell my character level RNN to generate a name of length 3, 4, 5 and so on. I haven't found anything online like this, but my ...
3
votes
0answers
83 views

Are there principled ways of tuning a neural network in case of overfitting and underfitting?

Whenever I tune my neural network, I usually take the common approach of defining some layers with some neurons. If it overfits, I reduce the layers, neurons, add dropout, utilize regularisation. ...
3
votes
1answer
215 views

Using tensor networks as machine learning models

Tensor networks (check this paper for a review) are a numerical method originally introduced in condensed matter physics to model complex quantum systems. Roughly speaking, such systems are described ...
3
votes
0answers
294 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
3
votes
0answers
41 views

What are some ways to quickly evaluate the potential of a given NN architecture?

Main question Is there some way we can leverage general knowledge of how certain hyperparameters affect performance, to very rapidly get some sort of estimate for how good a given architecture could ...

1
2 3 4 5
16