Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
4
votes
3answers
665 views

Do neurons of a neural network model a linear relationship/is it coincidence that the activation function looks like the general function of a line?

I'm certain that this is a very naive question, but I am just beginning to look more deeply at neural networks, having only used decision tree approaches in the past. Also, my formal mathematics ...
11
votes
2answers
574 views

Should deep residual networks be viewed as an ensemble of networks?

The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks: ...
1
vote
2answers
66 views

Divide classes into truncated and non-truncated objects

At the moment I am working on a vehicle counting & classification project. For a specific part in the project I need to get back only the completely visible vehicles from my input data (images). ...
8
votes
3answers
5k views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
3
votes
1answer
163 views

Does training happen during NEAT?

When one uses NEAT to evolve the best fitting network for a task, does training take place in each epoch as well? If I understand correctly, training is the adjustment of the weights of the neural ...
3
votes
0answers
43 views

Are there datasets to solve differential equations in a supervised fashion?

Are there datasets to solve differential equations in a supervised fashion? More precisely, the input is a differential equation and the label should be the general solution to that differential ...
10
votes
2answers
130 views

What does it mean for a neuron in a neural network to be activated?

I just stumbled upon the concept of neuron coverage, which is the ratio of activated neurons and total neurons in a neural network. But what does it mean for a neuron to be "activated"? I know what ...
0
votes
2answers
34 views

How do we choose the activation function for each hidden node? [duplicate]

I am new to neural networks. I would like to use them as a fitting or forecasting method. A simple NN model that does not contain hidden layers, that is, the input nodes are directly connected to the ...
2
votes
2answers
77 views

Why is MSE used over other quadratic loss functions?

So I was wondering, why I have only encountered square loss function also known as MSE. The only nice property of MSE I am so far aware of is its convex nature. But then all equations of the form $x^{...
0
votes
1answer
22 views

Is this learning rate schedule increasing the learning rate?

I was reading a PyTorch code then I saw this learning rate scheduler: ...
27
votes
9answers
5k views

Is artificial intelligence vulnerable to hacking?

The paper The Limitations of Deep Learning in Adversarial Settings explores how neural networks might be corrupted by an attacker who can manipulate the data set that the neural network trains with. ...
2
votes
1answer
12 views

Independance of Gradient of feedforward neural network

Can the gradient of a feedforward neural network at a layer be assumed to be independent of the activations of the previous layers ? I read this in a paper titled "Mean Field Residual Networks: On the ...
3
votes
1answer
27 views

Any explanation why multiple linear layers work better than a single linear layer in practice?

It is a well-known math fact that composition of linear/affine transformations is still linear/affine. For a naive example, $\textbf{A}_1\textbf{A}_2\textbf{x}$ is simply $\textbf{A}\textbf{x}$ ...
1
vote
0answers
9 views

How to choose the suitable Neural Network Architecture for Regression Tasks

so I'm working on a Project where I want to predict the Vehicle Position from the Vehicle Data like speed, acceleration etc.. now the data that I have comes also with a timestamp for each sample ( I ...
0
votes
0answers
50 views

How to develop face recongiton program using CNN to obtain more than 95% accuracy? [on hold]

I want to develop face recognition program using convolutional neural network. Can some one tell me steps to follow to do the same? I am new to deep learning. I want to develop it on windows using ...
3
votes
2answers
62 views

Detecting Sine Waves with a Neural Network?

I am recording the vibrations of an AC Motor (50Hz Europe) and I am trying to find out whether it is powered on or not. When I record these vibrations, I basically get the vibration values (-1 to +1) ...
0
votes
0answers
7 views

Multiple digits classifier using MobileNetV1

I want to train a model to be able to classify 16 number of digits in image, the numbers of digits are constants. So I create my dataset like these images (Print random 16 number with different font ...
2
votes
1answer
40 views

How can I teach a computer to play N64 games using Neural Nets?

I would like to work on a project where I teach an NN to play N64 games. To my current understanding, I would need an emulator? I can do the Machine Learning side of it, im just unsure how I can ...
1
vote
1answer
132 views

Which machine learning models are universal function approximators?

The universal approximation theorem states that a feed-forward neural network with a single hidden layer containing a finite number of neurons can approximate a wide variety of interesting (...
4
votes
2answers
59 views

What is the benefit of using identity mapping layers in deep neural networks like ResNet?

As I understand Resnet has some identity mapping layers that their task is to create the output as the same as the input of the layer. the resnet solved the problem of accuracy degrading. But what is ...
2
votes
2answers
76 views

Which neuron represents which part of the input?

In a neural network, each neuron represents some part of the input. For example, in the case of a MNIST digit, consider the stem of the number 9. Each neuron in the NN represents some part of this ...
1
vote
1answer
55 views

What order should I learn about Neural Networks?

I'm interested in learning about Neural Networks and implementing them. I'm particularly interested in GANs and LSTM networks. I understand perceptrons and basic Neural Network configuration (sigmoid ...
2
votes
1answer
33 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
0
votes
0answers
22 views

How to calculate covariance matrix of the mini-batch in the k-th layer using Python?

I am a beginner in Python. I want to calculate the covariance matrix of a mini-batch in a given hidden layer.
1
vote
0answers
16 views

How to train a transformer text to text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
1
vote
0answers
15 views

Trying to separate spiral data with neural network, learning tensorflow

I am learning how to use tensorflow without keras, just to make sure I understand tensorflow directly. I created a spiral-looking datasets with 100 points of each class (200 total), and I created a ...
0
votes
0answers
14 views

How to recognize two different objects with the similar shape, but different size

I am using Mask-RCNN neural network. I retrained my network to detect and mask wheels of die-cast toy cars. I am using images, which present the side of the car (left or right). Sometimes the cars ...
4
votes
1answer
2k views

Several questions regarding the NEAT algorithm

I've recently read the paper Evolving Neural Networks through Augmenting Topologies which introduces NEAT. I am now trying to prototype it myself in JavaScript. However, I stumbled across a few ...
3
votes
2answers
156 views

Handling emotion in informal text (Hi vs HIIIIII!!!!)?

This is a question related to Neural network to detect "spam"?. I'm wondering how it would be possible to handle the emotion conveyed in text. In informal writing, especially among a ...
0
votes
0answers
58 views

What to do when an image classifier does good for a class but bad for another?

So I wrote a convolutional neural network for a binary image classification. I have around 5300 images for each class which I thought would be enough to at least give me a good accuracy on the ...
4
votes
1answer
30 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
12
votes
5answers
18k views

Why does C++ seem less widely used in AI?

I just want to know why do Machine Learning engineers and AI programmers use languages like python to perform AI task and not C++ even though C++ is technically a more powerful language than python.
29
votes
3answers
17k views

Why is Lisp such a good language for AI?

I've heard before from computer scientists and from researchers in the area of AI that that Lisp is a good language for research and development in artificial intelligence. Does this still apply, with ...
1
vote
2answers
87 views

What are neural networks and how do they relate to AI?

Artificial neural networks (ANN) are computing systems vaguely inspired by the biological neural networks that constitute animal brains, how do they relate to AI?
3
votes
1answer
43 views

Is it possible to do K-nearest-neighbours before training DNN

The following X-shape alternated pattern can be separated quite well and super fast by K-nearest Neighbour algorithm (go to http://ml-playground.com to test it): However, DNN seems to face great ...
5
votes
2answers
89 views

What is the tolerance level of Standard-deviation of ANNs accuracy?

Just working with fully connected NNs (supervised learning), I found that models trained for, say NLP, on identical data sets with identical parameters to algorithms; but at different times, can ...
1
vote
1answer
42 views

What are the differences between Bytenet and Wavenet?

I recently read Bytenet and Wavenet and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the ...
7
votes
1answer
437 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
6
votes
1answer
107 views

Synapses automatically select it's neurons

I know the basics of Artificial Neural Networks. For instance; make dot product with the weights and every neuron from previous layer. Adjust the weight by error. And done, That is how I see neural ...
1
vote
1answer
144 views

Should I apply ReLU to non negative output?

Suppose I want to predict the position of a sensor based on its reading. I can first predict the unit vector and predict the distance to be multiplied to this vector. And I know that distance will ...
3
votes
1answer
49 views

What is the best variant of darknet to use?

pjreddie's official darknet version (link from official website here) has been forked several times. In particular I've come accross AlexeyAB's fork through this tutorial. I assume the tutorial's ...
4
votes
1answer
43 views

How to deal with large (or NaN) neural network's weights?

My weights go from being between 0 and 1 at initialisation to exploding into the tens of thousands in the next iteration. In the 3rd iteration they become so large that only arrays of nan values are ...
4
votes
1answer
43 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
5
votes
0answers
21 views

What is the impact of using multiple BMUs for self-organizing maps?

Sort of a conceptual question here. I was implementing a SOM algorithm to better understand its variations and parameters and got curious about one bit: the BMU (best matching unit == the neuron that ...
5
votes
2answers
119 views

Why do layered neural nets struggle with continous data?

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. It was my understanding that this was the ...
11
votes
1answer
176 views

What are all the different kinds of neural networks used for?

I found the following neural network cheat sheet (Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data). What are all these different kinds of neural networks used ...
3
votes
0answers
22 views

Vector normalization by a neural network

I'm wondering if there is a NN that can achieve the following task: Output a unit vector that is parallel to the input vector. i.e., input a vector $\mathbf{v}\in\mathbb{R}^d$, output $\mathbf{v}/\|\...
3
votes
2answers
98 views

What kind of neural network architecture is suitable for variable length block-like time series data?

I'm not sure what this type of data is called, so I will give an example of the type of data I am working with: A city records its inflow and outflow of different types of vehicles every hour. More ...
0
votes
2answers
41 views

How should I interpret the weights file of the Leela Zero neural network?

I am trying to understand the NN architecture given at https://github.com/leela-zero/leela-zero/blob/next/training/caffe/zero.prototxt. So, I downloaded the NN weights from http://zero.sjeng.org/. ...
3
votes
2answers
62 views

Translating a single word Neural Networks

I've been given an assignment to create a neural network that will suggest a Croatian word for a word given in any other European language (out of those found here). The words are limited to drinks ...