Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
2
votes
2answers
70 views

In layman terms, what does "attention" do in a transformer?

I heard from many people about the paper titled Attention Is All You Need by Ashish Vaswani et al. What actually does the "attention" do in simple terms? Is it a function, property, or some ...
0
votes
0answers
9 views

What is uncentered variance and how it becomes equal to mean square in Adam?

I have been reading about Adam and AdamW (Here). The author mentioned that in "uncentered variance" we don't consider subtracting mean In this statement, the author is talking about ...
0
votes
0answers
14 views

How to train network with multiprocessing?

I am trying to figure out how multiprocessing works in neural networks. In the example I've seen, the database is split into $x$ parts (depending on how many workers you have) and each worker is ...
0
votes
2answers
101 views

How to extract the main text from a formatted text file?

My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output. Take the scenario: One prints a PDF file to a text file (...
0
votes
0answers
29 views

Hand Landmark Detector Not Converging

I'm currently trying to train a custom model with TensorFlow to detect 17 landmarks/keypoints on each of 2 hands shown in an image (fingertips, first knuckles, bottom knuckles, wrist, and palm), for ...
4
votes
1answer
131 views

What is the cost function of a transformer?

The paper Attention Is All You Need describes the transformer architecture that has an encoder and a decoder. However, I wasn't clear on what the cost function to minimize is for such an architecture. ...
1
vote
0answers
12 views

When is an object detection approach over a CNN approach appropriate?

I understand that CNNs are for image classification while object detection is for localization + classification of the objects detected. However, in particular, AI for chest radiographs, why is object ...
0
votes
0answers
21 views

In mini-batch gradient descent, are the weights updated after each batch or after all the batches have gone through an epoch?

Say I have a mini-batch of size 32, and I have 10 such batches. Assuming I only run it for one epoch (just for the sake of understanding it), Will the weights be updated using the gradients of one ...
1
vote
1answer
16 views

Can I use the transformers for the prediction of historical data?

Can I use the transformers for the prediction of wind power with the historical data? Dataset Datetime, Ambient temperature (Degree), Dewpoint (Degree), Relative Humidity\n (%), Air Pressure, Wind ...
1
vote
1answer
56 views
+100

Closed networks vs Networks with a removed delay to predict new data

I've come across two types of neural networks to predict, both from Matlab, the closed structure and the net that removes one delay to find new data. From Matlab's app generated scripts we see: % ...
0
votes
1answer
77 views

Predicting probabilities of events using neural networks

I've got a few thousands of sequences like 1.23, 2.15. 3.19, 4.30, 5.24, 6.22 where the numbers denote times on which an event happened (there's just a single ...
1
vote
0answers
13 views

Should I allow NN to infer relationships of inputs?

This question is assuming a sequential, deep neural network Given some features [X1, X2, ... Xn], I'm trying to predict some value Y. The raw data available to me contains feature ...
0
votes
1answer
77 views

How to make input variable as trainable parameter in a neural network?

I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given ...
4
votes
2answers
648 views

Do we need automatic hyper-parameter tuning when we have a large enough dataset?

Hyperparameter tuning is the process of selecting the optimal hyperparameters for an ANN. Now, my guess is that, if we have sufficient data (say, 1.4 million for, say, 6 features), the model can be ...
10
votes
3answers
3k views

Why exactly do neural networks require i.i.d. data?

In reinforcement learning, successive states (actions and rewards) can be correlated. An experience replay buffer was used, in the DQN architecture, to avoid training the neural network (NN), which ...
0
votes
0answers
17 views

Sound in opencv [closed]

I have written opencv code using python, the code is fueled with a deep learning model that detects hand gesture and extracts meaning "sign language" I was able to extract the meaning of the ...
3
votes
2answers
708 views

How can I "measure" an object using Computer Vision techniques and neural networks?

I would like to develop a neural network to measure the distance between two opposite sides of an object in an image (in a similar way that the fractional caliper tool measures an object). So, given ...
0
votes
0answers
17 views

pytorch TypeError: forward() takes 1 positional argument but 2 were given [closed]

I have been trying to implement a small VGG network but run into this error. Here is the error message I am getting: ...
4
votes
1answer
193 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in The Unreasonable Effectiveness of Recurrent Neural Networks) that when training RNN on text for something like language modeling, the text is usually featurized character-by-...
3
votes
1answer
225 views

Using tensor networks as machine learning models

Tensor networks (check this paper for a review) are a numerical method originally introduced in condensed matter physics to model complex quantum systems. Roughly speaking, such systems are described ...
1
vote
0answers
32 views

Different equations for Yolov3 in courses/ articles and Darknet GitHub code?

I am confused by the equations for bounding boxes I find online. Some articles say that box_width = anchor_width * exp(residual_value_of_box_width)) and the ...
1
vote
0answers
8 views

References for the convergence of gradient-based algorithms for training neural networks

I'm looking for some good references that give convergence results of training neural networks. I'm decently familiar with works that analyze the convergence of SGD, and, in particular, I really like ...
4
votes
1answer
156 views

How to improve the performance of my model trained with NEAT for a drone to learn how to fly?

I am working on a project in which a drone needs to learn how to fly. I am using NEAT. For the first experiment, I want the drone to learn how to hover inside a $3 \times 3 \times 3$ meters box. My ...
6
votes
1answer
93 views

How does the network know which objects to track in the paper "Label-Free Supervision of Neural Networks with Physics and Domain Knowledge"?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
0
votes
1answer
63 views

How do I prepare this 3D data for NN?

How do I prepare the info of 3D models to use with NN? For example, I have thousands of models with boxes similar to the ones in the image below. I can extract the vertices and their normals that make ...
2
votes
1answer
256 views

Why am I getting a difference between training accuracy and accuracy calculated with Keras' predict_classes on a subset of the training data?

I'm trying to solve a binary classification problem with AlexNet. I split the original dataset into training and validation datasets using a 70/30 ratio. I have trained my neural network with a ...
0
votes
1answer
122 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
1
vote
1answer
827 views

Can gradient descent training be used for non-smooth loss functions?

I have non-smooth loss function $f(x) = \min(x, 0.5)$. Can gradient descent be used for training neural networks with such functions? Can gradient descent be used for fairly general, mathematically ...
2
votes
1answer
199 views

Can particle swarm optimization be used to train neural networks with more than one hidden layer?

I've been thinking about the idea of replacing the classic gradient descent algorithm with an algorithm that is less sensitive to a local optimum. I was thinking about particle swarm optimization (PSO)...
1
vote
1answer
2k views

What’s the difference between LSTM and RNN?

What's the difference between LSTM and RNN? I know that RNN is a layer used in neural networks, but what exactly is an LSTM? Is it also a layer with the same characteristics?
3
votes
2answers
1k views

Which model should I use to find (only) the object location (in terms of coordinates) in an image?

I am generating images that consist of points, where the object's location is where the most overlap of points occurs. In this example, the object location is $(25, 51)$. I am trying to train a model ...
1
vote
0answers
20 views

Should I train a neural network with data with or without a constraint?

I want to train a Neural Network (NN) using a dataset. I want to use the NN model as a prediction function in one algorithm. However, in the algorithm, any data that does not meet a specific ...
0
votes
0answers
15 views

What is the effect of gradient clipping by norm on the performance of a model?

It is recommended to apply gradient clipping by normalization in case of exploding gradients. The following quote is taken from here answer One way to assure it is exploding gradients is if the loss ...
2
votes
2answers
32 views

Improving validation losses and accuracy for 3D CNN

I have used a 3D CNN architecture, for detecting the presence of a particular promoter (MGMT), by using FLAIR brain scans. (64 slices per patient). The output is supposed to be binary (0/1). I have ...
0
votes
0answers
7 views

Why does conditioning neural network function on adjacency matrix of graph allow for distribution of gradient information from the supervised loss?

I was reading the following paper here and had a question about the paragraph on page 1 (in the introduction). The equation being referred to is: $$ \mathcal{L} = \mathcal{L}_0 + \lambda \mathcal{L}_{\...
6
votes
1answer
56 views

What is the impact of using multiple BMUs for self-organizing maps?

Here's a sort of a conceptual question. I was implementing a SOM algorithm to better understand its variations and parameters. I got curious about one bit: the BMU (best matching unit == the neuron ...
0
votes
1answer
63 views

Backpropagation after N sequential input-output pass

I'm trying to train a Neural Network in a particular situation -- similar to a genetic algorithm domain as far as I know. I have to run a simulation with a length of $K$ steps. I have a neural network ...
0
votes
0answers
13 views

Best way to measure regression accuracy?

I'm asking because classification problems have very concrete metrics like accuracy that are totally transparent to understand. Whereas regression models seem to have a very large number of possible ...
1
vote
1answer
27 views

What is the best open source python repo for facial recognition? [closed]

I am looking for best open source python repo for facial recognition. Best if it uses tensorflow backend. I know you can train images to recognize. Yolo can be used if trained on face. To name the ...
2
votes
1answer
344 views

How to detect multiple playing cards of the same class with a neural network?

I want to train an AI to detect the class (i.e. suit and rank) of playing cards. Playing cards from different decks may use slightly different shapes or colors to represent these attributes, and I ...
1
vote
0answers
39 views

Is stability an attribute of model or training algorithm used or combination of both?

From this answer, stability is attributed to a learning algorithm A stable learning algorithm is one for which the prediction does not change much when the training data is modified slightly. At ...
7
votes
2answers
678 views

How to deal with large (or NaN) neural network's weights?

My weights go from being between 0 and 1 at initialization to exploding into the tens of thousands in the next iteration. In the 3rd iteration, they become so large that only arrays of nan values are ...
8
votes
0answers
92 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
30
votes
5answers
21k views

Is it possible to train the neural network to solve math equations?

I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations? So given the ...
1
vote
0answers
23 views

How do you decide that you have tested enough hyper-parameter combinations for a specific neural network architecture?

How do you decide that you have tested enough hyper-parameter combinations for a specific neural network architecture to discard it and move on to a new model? Do you have a structured (generic) ...
0
votes
1answer
38 views

How can abstract graphs be recognized by neural nets?

Recognition of optical patterns (as pixel maps) by neural networks is standard. But optical patterns may be only slightly distorted or noisy, and may not be arbitrarily scrambled – e.g. by ...
1
vote
1answer
64 views

What are some suitable positive functions as activations of neural networks?

I am working on a deep Q-learning project. My project is different than normal deep Q-learning. The rewards of my neural network must be positive because I need their values to importance sample ...
0
votes
0answers
29 views

Understanding neural networks architecture visually

I am following this book and I am trying to visualize the network. This part seems tricky to me and I am trying to get my head around it by visualizing it: ...
1
vote
1answer
42 views

Predict time series from initial non-time dependant parameters

I'm trying to create an algorithm (neural network) that is able to predict a time series from a set of different parameters that are not given through time. Let's say I have a plane flying under the ...
0
votes
1answer
36 views

Does the ANN's training data include the proper output for every neuron?

I was designing an Artificial Neural Network a while back, but hit a bump when I got to the backpropagation. I was having trouble making the script choose whether to add or subtract from the weights, ...

1
2 3 4 5
45