Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

How to detect the sine wave signal with different frequency using neural networks?

I'm wondering if there is a way to use a neural network that can detect the noisy sine wave, where the frequency is not constant. In other words, I'm not looking for solution that would detect the ...
0
votes
0answers
13 views

What is uncentered variance and how it becomes equal to mean square in Adam?

I have been reading about Adam and AdamW (Here). The author mentioned that in "uncentered variance" we don't consider subtracting mean In this statement, the author is talking about ...
0
votes
0answers
18 views

How to train network with multiprocessing?

I am trying to figure out how multiprocessing works in neural networks. In the example I've seen, the database is split into $x$ parts (depending on how many workers you have) and each worker is ...
2
votes
2answers
82 views

In layman terms, what does "attention" do in a transformer?

I heard from many people about the paper titled Attention Is All You Need by Ashish Vaswani et al. What actually does the "attention" do in simple terms? Is it a function, property, or some ...
1
vote
1answer
19 views

Can I use the transformers for the prediction of historical data?

Can I use the transformers for the prediction of wind power with the historical data? Dataset Datetime, Ambient temperature (Degree), Dewpoint (Degree), Relative Humidity\n (%), Air Pressure, Wind ...
1
vote
0answers
13 views

Should I allow NN to infer relationships of inputs?

This question is assuming a sequential, deep neural network Given some features [X1, X2, ... Xn], I'm trying to predict some value Y. The raw data available to me contains feature ...
0
votes
0answers
31 views

Hand Landmark Detector Not Converging

I'm currently trying to train a custom model with TensorFlow to detect 17 landmarks/keypoints on each of 2 hands shown in an image (fingertips, first knuckles, bottom knuckles, wrist, and palm), for ...
1
vote
0answers
12 views

When is an object detection approach over a CNN approach appropriate?

I understand that CNNs are for image classification while object detection is for localization + classification of the objects detected. However, in particular, AI for chest radiographs, why is object ...
4
votes
2answers
651 views

Do we need automatic hyper-parameter tuning when we have a large enough dataset?

Hyperparameter tuning is the process of selecting the optimal hyperparameters for an ANN. Now, my guess is that, if we have sufficient data (say, 1.4 million for, say, 6 features), the model can be ...
0
votes
0answers
18 views

Sound in opencv [closed]

I have written opencv code using python, the code is fueled with a deep learning model that detects hand gesture and extracts meaning "sign language" I was able to extract the meaning of the ...
0
votes
0answers
21 views

In mini-batch gradient descent, are the weights updated after each batch or after all the batches have gone through an epoch?

Say I have a mini-batch of size 32, and I have 10 such batches. Assuming I only run it for one epoch (just for the sake of understanding it), Will the weights be updated using the gradients of one ...
0
votes
0answers
17 views

pytorch TypeError: forward() takes 1 positional argument but 2 were given [closed]

I have been trying to implement a small VGG network but run into this error. Here is the error message I am getting: ...
1
vote
0answers
8 views

References for the convergence of gradient-based algorithms for training neural networks

I'm looking for some good references that give convergence results of training neural networks. I'm decently familiar with works that analyze the convergence of SGD, and, in particular, I really like ...
0
votes
0answers
15 views

What is the effect of gradient clipping by norm on the performance of a model?

It is recommended to apply gradient clipping by normalization in case of exploding gradients. The following quote is taken from here answer One way to assure it is exploding gradients is if the loss ...
1
vote
0answers
32 views

Different equations for Yolov3 in courses/ articles and Darknet GitHub code?

I am confused by the equations for bounding boxes I find online. Some articles say that box_width = anchor_width * exp(residual_value_of_box_width)) and the ...
1
vote
0answers
20 views

Should I train a neural network with data with or without a constraint?

I want to train a Neural Network (NN) using a dataset. I want to use the NN model as a prediction function in one algorithm. However, in the algorithm, any data that does not meet a specific ...
1
vote
1answer
59 views

Closed networks vs Networks with a removed delay to predict new data

I've come across two types of neural networks to predict, both from Matlab, the closed structure and the net that removes one delay to find new data. From Matlab's app generated scripts we see: % ...
0
votes
0answers
7 views

Why does conditioning neural network function on adjacency matrix of graph allow for distribution of gradient information from the supervised loss?

I was reading the following paper here and had a question about the paragraph on page 1 (in the introduction). The equation being referred to is: $$ \mathcal{L} = \mathcal{L}_0 + \lambda \mathcal{L}_{\...
2
votes
2answers
32 views

Improving validation losses and accuracy for 3D CNN

I have used a 3D CNN architecture, for detecting the presence of a particular promoter (MGMT), by using FLAIR brain scans. (64 slices per patient). The output is supposed to be binary (0/1). I have ...
0
votes
0answers
13 views

Best way to measure regression accuracy?

I'm asking because classification problems have very concrete metrics like accuracy that are totally transparent to understand. Whereas regression models seem to have a very large number of possible ...
1
vote
1answer
27 views

What is the best open source python repo for facial recognition? [closed]

I am looking for best open source python repo for facial recognition. Best if it uses tensorflow backend. I know you can train images to recognize. Yolo can be used if trained on face. To name the ...
1
vote
0answers
23 views

How do you decide that you have tested enough hyper-parameter combinations for a specific neural network architecture?

How do you decide that you have tested enough hyper-parameter combinations for a specific neural network architecture to discard it and move on to a new model? Do you have a structured (generic) ...
0
votes
0answers
29 views

Understanding neural networks architecture visually

I am following this book and I am trying to visualize the network. This part seems tricky to me and I am trying to get my head around it by visualizing it: ...
1
vote
0answers
36 views

How does backpropagation know which weights to change?

I'm currently working on constructing a neural network from scratch (in JavaScript). I'm in the middle of working on the backpropagation, but there's something I don't understand: how does the ...
0
votes
1answer
36 views

Does the ANN's training data include the proper output for every neuron?

I was designing an Artificial Neural Network a while back, but hit a bump when I got to the backpropagation. I was having trouble making the script choose whether to add or subtract from the weights, ...
2
votes
0answers
29 views

Which method can accurately detect circular/angular shapes? (attached example)

Is there a method to detect shapes like these accurately and efficiently? I have tried the OpenCv Haar Casacade Classifier which does not work well. These shapes should all be the same class object ...
0
votes
1answer
31 views

Compare the efficiency of a trained ML model with a non-learning-based method for solving the same problem

If a certain task T is solved by a non-learning-based method A (let's say, an optimization-based approach). We now train a machine learning model B (let's say a neural network) on the same task. What ...
0
votes
1answer
29 views

How to pass multiple vectors and numeric features as input to the neural network?

I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
1
vote
0answers
27 views

FCNs: Questions about the filter rarefaction in the CVPR paper [Long et al., 2015]

I am reading the paper about the fully convolutional network (FCN). I had some questions about the part where the authors discuss the filter rarefaction technique (I guess this is roughly equivalent ...
0
votes
0answers
11 views

What is the difference between supervised and unsupervised training in T5?

I know unsupervised training for T5 is like: input: He went X output: X to school Z is this equivalent to the following in a supervised manner: ...
0
votes
0answers
31 views

Discrepencies between the TimeGan paper and the code?

I recently read the paper Time-Series Generative Neural Networks and found the results that they reported quite promising (https://proceedings.neurips.cc/paper/2019/file/...
1
vote
1answer
43 views

What is the difference between gradient decent in neural networks and temporal difference in reinforcement learning?

I am studying Q-learning in reinforcement learning. My question is about the Bellman equation. In Q-learning, the Bellman equation is often introduced as follows. \begin{align} Q_{new}(s,a) &= Q_{...
-1
votes
1answer
24 views

What practically makes a good architecture of ANN?

When we take a look at the literature there are so many opinions. I was wondering what are some generally good practices to design an architecture, like how much depth would you prefer and how much ...
2
votes
1answer
36 views

Does a bigger neural network learn "worse" representations than a small neural network when the amount of data isn't enough?

Assume we have a neural network and we want to train it on a classification problem. The hidden layers of the neural network are kind of feature representations of the input data. If the neural ...
0
votes
1answer
13 views

Should I train my network for classification on samples whose ground truth label is ambiguous?

Imagine that I am training a model to classify handwritten digits. Suppose there are some bad quality images that could be classified by a human as either 0 or 8, 1 or 7 or other commonly ...
1
vote
1answer
21 views

How many iterations of the optimisation algorithm are performed on each mini-batch in mini-batch gradient descent?

I understand the idea of mini-batch gradient descent for neural networks in that we calculate the gradient of the loss function using one mini-batch at a time and use this gradient to adjust the ...
0
votes
0answers
24 views

Training a neural network using several data sources with quality flags

I have been searching for a specific problem in training a NN and hope someone is able and willing to help as I cannot find a solution. The problems is that I have a spatial data set with 3 sources of ...
0
votes
2answers
32 views

In mini-batch gradient descent, do we pass each input in the batch individually or all inputs at the same time through the layer?

In the stochastic gradient descent algorithm, the weight update happens for every training sample. In the mini-batch gradient descent algorithm, the weight update happens for every batch of training ...
0
votes
2answers
38 views

Is there any relationship between the batch size and the number of epochs?

I am currently running a program with a batch size of 17 instead of batch size 32. The benchmark results are obtained at a batch size of 32 with the number of epochs 700. Now I am running with batch ...
2
votes
2answers
59 views

How general is generalization?

I am sorry but I have to explain my question using an example, I do not know how to ask it in proper scientific terms. Let's assume, I have trained a deep learning model on classifying hand gestures, ...
0
votes
0answers
13 views

Brain tumour detection using CNN

I have a fairly basic mathematical and implementational understanding of ML algorithms and CNNs, and I am trying to think of an approach for this task: https://www.kaggle.com/c/rsna-miccai-brain-tumor-...
0
votes
0answers
10 views

Usecases where pretrained models are used without retraining

I was starting out with deep learning and come across a lot of pretrained models in frameworks and sites such as tensorflow model zoo. Are these models actually used by other developers in real use ...
0
votes
0answers
26 views

Is this a good implementation of this LSTM architecture?

I had been looking at some OCR problems and came across this presentation. I implemented it. In the presentation, there is the LSTM-Stack (diagram and algorithm, slide 32): Here is a visualization of ...
1
vote
1answer
54 views

Is "width of a neural network" a wrong phrase?

Depth of the neural network is equal to the total number of layers in the neural network except input layer. so, neural network with more number of layers are called deep neural networks. Width, in ...
0
votes
0answers
20 views

Multiple GRU layers to improve a text generation

I am using the model in this colab https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb#scrollTo=AM2Uma_-yVIq for Shakespeare like text generation. ...
0
votes
1answer
72 views

Why is the validation accuracy lower in case of CNN?

I fed the same set of 1.4 million data to two different models: MLP CNN model In both cases, I used the same parameters and hyperparameters. The CNN is showing comparatively lower accuracy (80%) ...
0
votes
1answer
27 views

How to handle random order of inputs and get same output?

I am a beginner with DL. I did some tutorials and I know the basics of TensorFlow. But I have a problem understanding how to construct more advanced NNs. Let's say I have 6 inputs and a list of 500 ...
2
votes
1answer
62 views

What problem does the neural network really solve?

In the image below taken from a Youtube video, the author explains that the neural network can be used to fit a relational graph for a set of data points shown by the green line. And that this is ...
0
votes
0answers
20 views

NLP problem Phrase/Token labeling

Looking for suggestions on how to define the following NLP problem and different ways in which it can be modeled to leverage machine learning. I believe there are multiple ways to model this problem. ...
0
votes
0answers
31 views

What deep reinforcement learning algorithm should I use for my problem?

So here is a description of my problem: Essentially, I have a large amount of files filled with code for a number of different tasks. However, lets say these codes are inefficient, and should be ...

1
2 3 4 5
45