Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
4
votes
3answers
552 views

Do neurons of a neural network model a linear relationship?

I'm certain that this is a very naive question, but I am just beginning to look more deeply at neural networks, having only used decision tree approaches in the past. Also, my formal mathematics ...
0
votes
2answers
31 views

How do we choose the activation function for each hidden node?

I am new to neural networks. I would like to use them as a fitting or forecasting method. A simple NN model that does not contain hidden layers, that is, the input nodes are directly connected to the ...
0
votes
1answer
22 views

Is this learning rate schedule increasing the learning rate?

I was reading a PyTorch code then I saw this learning rate scheduler: ...
2
votes
1answer
11 views

Independance of Gradient of feedforward neural network

Can the gradient of a feedforward neural network at a layer be assumed to be independent of the activations of the previous layers ? I read this in a paper titled "Mean Field Residual Networks: On the ...
3
votes
1answer
27 views

Any explanation why multiple linear layers work better than a single linear layer in practice?

It is a well-known math fact that composition of linear/affine transformations is still linear/affine. For a naive example, $\textbf{A}_1\textbf{A}_2\textbf{x}$ is simply $\textbf{A}\textbf{x}$ ...
1
vote
0answers
9 views

How to choose the suitable Neural Network Architecture for Regression Tasks

so I'm working on a Project where I want to predict the Vehicle Position from the Vehicle Data like speed, acceleration etc.. now the data that I have comes also with a timestamp for each sample ( I ...
3
votes
2answers
62 views

Detecting Sine Waves with a Neural Network?

I am recording the vibrations of an AC Motor (50Hz Europe) and I am trying to find out whether it is powered on or not. When I record these vibrations, I basically get the vibration values (-1 to +1) ...
0
votes
0answers
50 views

How to develop face recongiton program using CNN to obtain more than 95% accuracy? [on hold]

I want to develop face recognition program using convolutional neural network. Can some one tell me steps to follow to do the same? I am new to deep learning. I want to develop it on windows using ...
0
votes
0answers
7 views

Multiple digits classifier using MobileNetV1

I want to train a model to be able to classify 16 number of digits in image, the numbers of digits are constants. So I create my dataset like these images (Print random 16 number with different font ...
2
votes
1answer
39 views

How can I teach a computer to play N64 games using Neural Nets?

I would like to work on a project where I teach an NN to play N64 games. To my current understanding, I would need an emulator? I can do the Machine Learning side of it, im just unsure how I can ...
2
votes
0answers
40 views

Are there datasets to solve differential equations in a supervised fashion?

Are there datasets to solve differential equations in a supervised fashion? More precisely, the input is a differential equation and the label should be the general solution to that differential ...
4
votes
2answers
59 views

What is the benefit of using identity mapping layers in deep neural networks like ResNet?

As I understand Resnet has some identity mapping layers that their task is to create the output as the same as the input of the layer. the resnet solved the problem of accuracy degrading. But what is ...
1
vote
0answers
16 views

How to train a transformer text to text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
1
vote
0answers
15 views

Trying to separate spiral data with neural network, learning tensorflow

I am learning how to use tensorflow without keras, just to make sure I understand tensorflow directly. I created a spiral-looking datasets with 100 points of each class (200 total), and I created a ...
0
votes
0answers
14 views

How to recognize two different objects with the similar shape, but different size

I am using Mask-RCNN neural network. I retrained my network to detect and mask wheels of die-cast toy cars. I am using images, which present the side of the car (left or right). Sometimes the cars ...
0
votes
0answers
22 views

How to calculate covariance matrix of the mini-batch in the k-th layer using Python?

I am a beginner in Python. I want to calculate the covariance matrix of a mini-batch in a given hidden layer.
1
vote
1answer
55 views

What order should I learn about Neural Networks?

I'm interested in learning about Neural Networks and implementing them. I'm particularly interested in GANs and LSTM networks. I understand perceptrons and basic Neural Network configuration (sigmoid ...
3
votes
1answer
49 views

What is the best variant of darknet to use?

pjreddie's official darknet version (link from official website here) has been forked several times. In particular I've come accross AlexeyAB's fork through this tutorial. I assume the tutorial's ...
3
votes
1answer
43 views

Is it possible to do K-nearest-neighbours before training DNN

The following X-shape alternated pattern can be separated quite well and super fast by K-nearest Neighbour algorithm (go to http://ml-playground.com to test it): However, DNN seems to face great ...
0
votes
0answers
58 views

What to do when an image classifier does good for a class but bad for another?

So I wrote a convolutional neural network for a binary image classification. I have around 5300 images for each class which I thought would be enough to at least give me a good accuracy on the ...
5
votes
0answers
21 views

What is the impact of using multiple BMUs for self-organizing maps?

Sort of a conceptual question here. I was implementing a SOM algorithm to better understand its variations and parameters and got curious about one bit: the BMU (best matching unit == the neuron that ...
4
votes
1answer
30 views

dimensions of hidden layer and cell state layer in LSTM

I was following some examples to get familiar with tensorflow LSTM related api, but noticed that all LSTM initialization functions require only num_units parameter which denotes number of hidden units ...
3
votes
0answers
22 views

Vector normalization by a neural network

I'm wondering if there is a NN that can achieve the following task: Output a unit vector that is parallel to the input vector. i.e., input a vector $\mathbf{v}\in\mathbb{R}^d$, output $\mathbf{v}/\|\...
4
votes
1answer
43 views

How to deal with large (or NaN) neural network's weights?

My weights go from being between 0 and 1 at initialisation to exploding into the tens of thousands in the next iteration. In the 3rd iteration they become so large that only arrays of nan values are ...
3
votes
1answer
32 views

GPU/TPU acceleration for neural networks with various network topologies

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
11
votes
1answer
175 views

What are all the different kinds of neural networks used for?

I found the following neural network cheat sheet (Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data). What are all these different kinds of neural networks used ...
3
votes
2answers
93 views

What kind of neural network architecture is suitable for variable length block-like time series data?

I'm not sure what this type of data is called, so I will give an example of the type of data I am working with: A city records its inflow and outflow of different types of vehicles every hour. More ...
0
votes
0answers
13 views

Predict the next best action based on previous lists of actions

I have the following problem. There is a software that I've written some time ago. Users enter customer's data in the system and there is a limited number of things (actions) that they can do/add - ...
2
votes
1answer
30 views

What sort of Neural Network is best suited to predicting a future purchase?

I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4. Now I'm trying to do a NN that can make a prediction. ...
-1
votes
0answers
48 views

Is My Formula For a Neural Network correct?

I am creating a multi-layer neuron library in C# and doubtful of my understanding of neural network correctness as my answer even after the training of xor is always nearer 0.5. Here are the notation ...
0
votes
0answers
17 views

Employing an Agent to Learn through Undirected Exploration?

So, I’m looking to automate some tasks at my job. I work at an engineering company. One of my tasks is produce these “reports” in excel that track some design metrics in our company. It’s a soul ...
2
votes
1answer
25 views

Neural network does not give out the required out put?

Made a neural network using tensor flows that was supposed matches an Ip to one of the 7 type of vulnerabilities and gives out what type of vulnerability that IP has. ...
3
votes
2answers
32 views

How can we find find the input image which maximizes the class-probability for an ANN?

Let's assume we have an ANN which takes a vector $x\in R^D$, representing an image, and classifies it over two classes. The output is a vector of probabilities $N(x)=(p(x\in C_1), p(x\in C_2))^T$ and ...
5
votes
1answer
74 views

Why evolutionary training of neural networks is not popular?

Evolutionary algorithms are mentioned in some sources as possible to be used to train a neural network (finding weights, not hyperparameters), however I have not heard about one practical application ...
1
vote
0answers
24 views

How to get same accuracy with identical models in Keras and Tensorflow?

As we all know Keras backend uses Tensorflow and so it should give out same kind of results when we provide same parameters, hyper-parameters, weights and biases initialisation at each layer, but ...
4
votes
1answer
43 views

How does the network know which objects to track in the paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge”?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
3
votes
1answer
30 views

Query on another perspective on Deep Learning

At least at some level, maybe not end-to-end always, but Deep Learning always learns a function, essentially a mapping from a Domain to a Range. The Domain and Range, at least in most cases, would be ...
4
votes
0answers
29 views

Get the position of an object, out of an image

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised ...
0
votes
0answers
17 views

A2C for the game of Hanabi underfits

I am trying to solve the game of Hanabi (paper describing game) with actor-critic algorithm. I took code for the environment from the Deepmind's repository and implemented a2c algorithm myself. From ...
0
votes
0answers
26 views

Neural Network training on one example to try overfitting leads to strange predictions

tldr; if I train the network on 1 training example, the outcome sometimes makes no sense at all, sometimes is as expected. If I train it on more examples and higher iterations, the network, which ...
2
votes
1answer
38 views

How do policy gradients compute an infinite probability distribution from a neural network

Do neural networks compute the probability distribution for policy gradient methods. If so, how do they compute an infinite probability distribution? How do you represent a continuous action policy ...
3
votes
1answer
22 views

Image classification with an associated matrix

I have a dataset of images with 9 different classes. However, there are different categories with the same type of associated image and only can be differentiated with an associated matrix in my ...
3
votes
0answers
19 views

Handwritten digits recognition during the process of writing

I know how to train a NN for recognizing handwritten digits (e.g. using the MNIST database). I'm wondering how to accomplish the same "online", which is during the process of writing e.g. I'm writing ...
1
vote
2answers
36 views

How to express accuracy of a regression ANN that uses MSE loss function?

I have a regression MLP network with all input values between 0 and 1, and am using MSE for the loss function. The minimum MSE over the validation sample set comes to 0.019. So how to express the '...
1
vote
0answers
12 views

CBIR Evaluation on contextually different data

How good would a CBIR system trained on a dataset, for example, DELF, trained on the Google Landmarks dataset, perform when evaluated on a contextually different dataset such as the WANG or the COREL ...
3
votes
1answer
109 views

Why isn't my Neural Network based calculator working?

I am playing around with neural networks in Tensorflow and I figured an interesting test would be whether I can write a calculator using a Tensorflow Neural Network. I started with simple addition ...
0
votes
1answer
62 views

How can the derivative of a neural network be calculated, given no mathematical expression?

Neural networks (NNs) are used as approximators in reinforcement learning (RL). To update the policy in RL, the actor network's gradients w.r.t its weights are needed. Since NN doesn't have a ...
1
vote
0answers
29 views

why the sigmoid function will be 1 and 0 if we use a fully connected layer that produce a big enough positive(res negative )output

HI I am using a fully connected network that uses sigmoid if we feed a a big enough weights the sigmoid function will finally become 1 or 0 , is there any solution to avoid this ? and will this lead ...
0
votes
0answers
38 views

Can neural networks output the rules it is using?

I have made a neural network using TensorFlow that is able to identify IP addresses that are likely to have a vulnerability of type A. I want to output the rule it has made for this identification.
0
votes
1answer
38 views

How many hidden layers are needed for this training data set

I'm trying to separate classes in 3D space, the data are as in the sketch below: There are 3 classes: 0,1,2; and with the look into the sketch, it seems that I need 3 planes to separate the classes, ...