Questions tagged [training]

For questions about training networks, rules systems, or other AI system components.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Loss for rnn training for text generation - more details about losses

Work is based on this colab https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb#scrollTo=vkvUIneTFiow Looking for more explanation on different ...
0
votes
0answers
15 views

Additional Training of TadGAN Neural Network Model

Background I am developing a TadGAN model (based off this paper), and am trying to learn how to feed subsequent time series data to the model such that the new data is additional training for the ...
0
votes
1answer
100 views

Mapping input vectors of variable length to output vectors of variable lengths with dummy variables

I have a general question about supervised ANNs that map inputs to outputs. It is possible to vary the length of the input and output vectors by inserting some dummy variables that will not be ...
1
vote
1answer
41 views

What problem does the neural network really solve?

In the image below taken from a Youtube video, the author explains that the neural network can be used to fit a relational graph for a set of data points shown by the green line. And that this is ...
1
vote
3answers
110 views

How to improve neural network training against a large data set of points with varying magnitude

I am currently using TensorFlow and have simply been trying to train a neural network directly against a large continuous data set, e.g. $y = [0.014, 1.545, 10.232, 0.948, ...]$ corresponding to ...
2
votes
0answers
167 views

How does one continue the pre-training in BERT?

I need some help with continuing pre-training on Bert. I have a very specific vocabulary and lots of specific abbreviations at hand. I want to do an STS task. Let me specify my task: I have domain-...
0
votes
1answer
31 views

How to re-training an AI model to have smaller input image size

I need a PyTorch Model which can do road segmentation on OAK-D camera. The model provided requires Input Image Size: 896*512, which is too big for running on OAK-D camera. Thus I need to re-training ...
1
vote
1answer
83 views

Why do the training and validation loss curves differge?

I was training a CNN model on TensorFlow. After a while I came back and saw this loss curve: The green curve is training loss and the gray one is validation loss. I know that before epoch 394 the ...
0
votes
0answers
24 views

What's the benefit for using a Kalman filter for training a neural network compared to other optimization algorithms?

I found a paper about using an Unscented Kalman Filter(UKF) for traning a neural network. The UKF filter is modified so it works for parameter estimation. Assume that we have a neural network model $\...
1
vote
1answer
132 views

If the i.i.d. assumption holds, shouldn't the training and validation trends be exactly the same?

If the i.i.d. (independent and identically distributed) assumption holds for a training-validation set pair, shouldn't their loss trends be exactly the same, since every batch from the validation set ...
0
votes
1answer
16 views

Validation accuracy less than training accuracy (with no sigh of overtraining)

I am working with a deep CNN with over 100k sample data. I divided it up into 75% training, 12.5% validation and 12.5% for testing. As I train my network, the training accuracy approaches near 100% ...
0
votes
0answers
18 views

How to handle critical points during generator training?

Using an MLP as generator introduces multiple critical points in parameter space. You can read this excerpt from research paper titled Generative Adversarial Nets In practice, adversarial nets ...
0
votes
1answer
26 views

Tensorflow object detection model total loss starts out good, but suddenly explodes up to high loss numbers

I'm training a Tensorflow object detection model with approx. 7500 images of two classes, which contains approx. 10,000 classes per class. I'm using Tensorflow 2.6.0, in case that is relavent. I am ...
0
votes
1answer
41 views

Does average loss function in GAN training is just an approximation of value function and does not ensure convergence of generator and discriminator?

The value function on which convergence has been proved by the original paper of GAN is $$\min_G \max_DV(D, G) = \mathbb{E}_{x ∼ P_{data}}[\log D(x)] + \mathbb{E}_{z ∼ p_z}[log (1 - D(G(z)))]$$ and ...
1
vote
2answers
91 views

What is Lipschitz constraint and why it is enforced on discriminator?

The following is the abstract for the research paper titled Improved Training of Wasserstein GANs Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training ...
-1
votes
1answer
110 views

Is training a neural network all about "balancing of weights and biases"?

I was trying to build an OCR system and heard about ANNs. I am weak at mathematics and statistics and couldn't stick up to reading those massive mathematical documents (research papers or ANN-related ...
0
votes
0answers
25 views

How big should the dataset for retraining ssd_mobilenet_v2 be?

I have retrained ssd_mobilenet_v2 using my own dataset with 2 classes (pen or pencil), using object detection API. For my project, I expect users to select specific pencils from all pencils and ...
1
vote
1answer
167 views

How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
1
vote
1answer
40 views

Can the optimal learning rate differ for different architectures?

In several courses and tutorials about neural networks, people often say that the learning rate (LR) should be the first hyper-parameter to be tuned before we tweak the others. For example, in this ...
7
votes
7answers
8k views

Why does training an SVM take so long? How can I speed it up?

I'm trying to create and test non-linear SVMs with various kernels (RBF, Sigmoid, Polynomial) in scikit-learn, to create a model which can classify anomalies and benign behaviors. My dataset includes ...
1
vote
0answers
32 views

Is stability an attribute of model or training algorithm used or combination of both?

From this answer, stability is attributed to learning algorithm A stable learning algorithm is one for which the prediction does not change much when the training data is modified slightly. At some ...
1
vote
1answer
21 views

Using Human Confirmation in place of a loss Function for Training

Has there been any experimentation in designing an AI to prompt a human to judge the accuracy of it's outcomes? instead of using a loss function, a human can judge the accuracy of it's estimation ...
1
vote
1answer
875 views

Add training data to YOLO post-training

(Cross-posting here from the data science stack exchange, as my question didn't get any replies. I hope it's okay!) I've been playing around with YOLOv3 and obtaining some good results on the ~20 ...
0
votes
0answers
18 views

Adding data to training results in loss random peaks

I have succesfully trained ssd_mobilenet_v2_keras for object detection, with a dataset of about 3700 images. Now I have more images to add. I tried adding only a few images (150-300) to see what ...
2
votes
1answer
97 views

Is there any way to train a neural network without using gradients?

The only algorithm I know for updation of weights of a neural network is based on gradients. The update equation can be roughly written as $$w \leftarrow w - \nabla_{w}L$$ where $\nabla_{w}L$ is the ...
3
votes
1answer
624 views

Periodic Pattern in Validation Loss Curve

I am currently trying to solve a regression problem using neural networks. I want to detect movement patterns in images over time (video) and output a continuous value. During the training process I ...
0
votes
0answers
11 views

What is meant by non-convergent limit cycles?

Limit cycle is a closed curve that is isolated i.e., no other closed curve near to it. You can read the following paragraph from here If there is (such) a closed curve, the nearby trajectories must ...
2
votes
1answer
44 views

What does it mean by strong or sufficient gradient for training in this context?

It has been mentioned in the research paper titled Generative Adversarial Nets that generator need to maximize the function $\log D(G(z))$ instead of minimizing $\log(1 −D(G(z)))$ since the former ...
0
votes
0answers
29 views

Why is the proof of convergence in the GAN paper not applicable practically?

This question is about generative adversarial networks and restricted to the research paper titled Generative Adversarial Nets. If I select a particular architecture of MLP as a generator and trained ...
0
votes
0answers
13 views

Weight for Samples on SVM (Support Vector Machine)

There is a option sample_weight in fit(X[, y, sample_weight]) function (OneClassSVM, sklearn library). If I use the option ...
0
votes
1answer
31 views

How to calculate the gradient penalty proposed in "Improved Training of Wasserstein GANs"?

The research paper titled Improved Training of Wasserstein GANs proposed a gradient penalty in order to avoid undesired behavior due to weight clipping of the discriminator. We now propose an ...
1
vote
0answers
22 views

Training on the dataset in parts vs training on the whole dataset

What is the difference between these two situations? are they the same ? #1 : train a model 20 epochs on the whole dataset #2 : divide dataset into n-parts then train the model 20 epochs on each part ...
2
votes
1answer
35 views

Data scan not making sense for coco dataset

I am doing a simple scan to see how dataset size affects training. Basically, I took 10% of the coco dataset and trained a yolov3 net (from scratch) to just look for people. Then I took 20% of the ...
0
votes
0answers
13 views

Feeding the output back to input in 3D CNN model

I am currently designing a Model which takes Input 3D Grid and Model Output at $t-1$. The model figure is described below I have two thoughts in training the model for above situation. Feed output $...
1
vote
0answers
19 views

Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?

Consider the following neural network with $\ell$ layers. $$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$ where $i, h, o$ stands for ...
1
vote
2answers
36 views

Train Validation Test Splitting After or Before Data Augmentation?

I have seen tutorials online saying that you should do data augmentation AFTER doing the train/val/test split. However, when I go online to read some research papers, I see numerous instances of ...
0
votes
0answers
4 views

How to estimate the number of training samples to train a HMM Model with Baum-Welch Algorithm

Is there a thumb rule that tells us how many data sample a HMM model needs to be trained by Baum-Welch Algorithm?
0
votes
2answers
112 views

What is the right way to train a generator in a GAN?

I am not fully understanding how to train a GAN's generator. I have a few questions below, but let me first describe what I am doing. I am using the MNIST dataset. I generate a batch of random images ...
1
vote
0answers
24 views

Is the main difference between the logistic regression and the perceptron the activation function they use?

I went through a Stats StackExchange's post about the difference between logistic regression and perceptron, which is too long to get the key point. I'd like to consider the question in terms of the ...
0
votes
0answers
29 views

Model not learning anything, what can be the problem?

I've trained a model for heart sound classification with transfer learning (MobileNet) on Physionet dataset, and it works fine. However, when I train it on my own dataset, it seems that it can not ...
2
votes
0answers
38 views

Identifying rotating and resizing letters with background noise

I'm trying to complete a captcha, and here is what it looks like: Between captchas the calligraphy of the letters is the same, but the letters may be resized and rotated. And the background noise (...
4
votes
1answer
98 views

REINFORCE algorithm for portfolio optimization - problem while training

I'm trying to implement the Reinforce algorithm (Monte Carlo policy gradient) in order to optimize a portfolio of 94 stocks on a daily basis (I have suitable historical data to achieve this). The idea ...
0
votes
0answers
56 views

How to validate a trained HMM Model?

What are some approaches to validate a HMM Model ? I want to check if a trained model is good enough to put it into operation or if it does need more training. I am using the Baum Welch algorithm to ...
6
votes
2answers
191 views

How to shorten the development time of a neural network?

I am developing an LSTM for sequence tagging. During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. After ...
3
votes
1answer
45 views

Can some of the weights be fixed during the training of a neural network?

Is it possible to exclude specific layers from the optimization? For example, let's say I have an input layer, 2 hidden layers, and the output layer. I know there is a perfect solution for my problem ...
2
votes
1answer
1k views

Should I continue training if the neural network attains 100% training accuracy?

I have a neural network where there are two hidden layers. Each hidden layer has 128 neurons. The input layer has 20 inputs, and the output layer has 3 outputs. I have 1 million records of data. 80% ...
1
vote
1answer
183 views

Why can't we train neural networks in a peer-to-peer manner?

I have recently been exposed to the concept of decentralized applications, I know that neural networks require a lot of parallel computing infra for training. What are the technical difficulties one ...
3
votes
1answer
482 views

How can I incrementally train a Yolo model without catastrophic forgetting?

I have successfully trained a Yolo model to recognize k classes. Now I want to train by adding k+1 class to the pre-trained weights (k classes) without forgetting previous k classes. Ideally, I want ...
1
vote
3answers
133 views

Is there a way of pre-determining whether a CNN model will perform better than another?

I developed a CNN for image analysis. I've around 100K labeled images. I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than ...

1
2 3 4 5
8