Questions tagged [training]
For questions about training networks, rules systems, or other AI system components.
510 questions
1
vote
1
answer
13
views
How do we call the technique of increasing the training set little by little during training?
When training a neural network in supervised learning, the common approach is to train on the whole training set.
However, it is technically possible to use only 50% of the training set, start ...
- 111
2
votes
1
answer
140
views
How does innovation deal with the problem of different connections in NEAT?
How does innovation deal with the problem of different connections in NEAT? An example makes it clearer: the neural network has two outputs 3 and 4, they are connected according to the scheme 10 -> ...
0
votes
0
answers
21
views
fine tuning Marbert for Tunisian dialect, ask for the tokenizer
i want to fine tune Marbert for tunisian text classification dialect, using this dataset : https://www.kaggle.com/datasets/waalbannyantudre/tunisian-arabizi-dialect-data-sentiment-analysis
i have test ...
0
votes
0
answers
17
views
How RNN layers see batch data during training?
I want to understand better step by step process how a single-batch data interacts with RNN (or LSTM) layer units for scalar time series.
I understand that RNNs are dense, meaning each unit (neuron) ...
- 1
1
vote
0
answers
17
views
What Policy/Agent and Observation Spec To Use For TensorFlow Agents For Video Game Platformer?
I'm trying to create a model to beat a video game platformer I made a few months ago. In the game, the platforms scroll down from the sky and the player has to keep jumping to them to avoid touching ...
- 11
2
votes
2
answers
80
views
Can data pollute a LLM?
I wonder this because well I am biased because I got a bad diagnosis and you would imagine what I think his science I consider, but this came to me yesterday while checking o1 in psychiatry topics and ...
user86598
2
votes
0
answers
29
views
How to analyze furniture for digital reconstruction?
My end-goal is to take a single photo of a piece of common furniture (couch, chair, table) and create a 3d model from that. I'm a novice with deep learning as I've only done basic CNN's and such with ...
- 119
0
votes
0
answers
19
views
My CNN validation Accuracy increases super slow?
im doing a retinopathy detection project with over 3500 images, 700 in each class. I've filtered the image like
It seems that my model isn't learning from the data, or is having trouble because the ...
- 1
0
votes
0
answers
48
views
Is it possible to train Llama 3.1, installed locally, on text and feed it with raw text to be changed?
I have installed Llama 3.1 locally following this tutorial. I want to feed it with some PDF files and a raw file. The PDF should serve as a training model to adapt the raw text file as necessary. Do ...
- 101
0
votes
2
answers
64
views
How do I improve my model accuracy and val_accuracy for my cnn model?
I'm using 3000+ retinopathy images in my CNN model. The accuracy remains around 77 to 80, how do i improve the accuracy value and reduce loss value?
I've tried dropout and Adam optimizer to increase ...
- 1
0
votes
0
answers
46
views
Early divergence of YOLOv7-tiny train and val obj_loss plots
I am training a YOLOv7-tiny model and have the following observations from the training session:
the train and val objectness loss plots diverged pretty early on in the training process
the class and ...
- 1
0
votes
0
answers
23
views
Moving tensor to cuda reduces precision
I am trying to move a tensor to cuda, but unfortunately, I am loosing precision. The first entry is 5.02e-5 which is converted to 0. Is there a work around?
...
- 1
0
votes
0
answers
18
views
Monitoring validation loss with confident mispredictions
I am working on a binary classification task using a variant of ResNet. The dataset consists of medical recordings and is relatively small (N=2000), though I apply various validated data augmentation ...
- 101
0
votes
0
answers
31
views
Creating a Robot Navigation Landscape from Vision and Motion
I have a robot that moves randomly around an apartment using two wheels. Each wheel can be powered independently: applying the same power to both wheels makes the robot move forward, while applying ...
0
votes
1
answer
45
views
When to stop a DQN agent train?
Hello AI Stack people,
I'm in doubt to when i should stop my DQN agent train.
I'm traning a DQN agent and i will use a hyperparameter optimization method (probabily random search). So, i need a ...
0
votes
1
answer
47
views
How to re-train ML model
I'm somewhat new to machine learning so there is one thing that I wanted to clarify.
I was playing around with AWS Sagemaker, trained a model with some labeled data, deployed it to endpoint and set up ...
- 103
1
vote
1
answer
19
views
Strange Periodic Train Accuracy During Toy LLM Pretraining
dear community,
I am trying to reproduce the result of Allen-Zhu's Physics of Language Model paper 3.1 (https://arxiv.org/abs/2309.14316). This paper is mainly about training a toy GPT-2 model on ...
- 31
6
votes
3
answers
1k
views
Why doesn't SiLU suffer from a worse version of a "dying ReLU" problem?
Unlike ReLU, the derivative of SiLU is non-zero everywhere except at the global minimum. However, intuitively it seems like having a negative gradient when the input is very negative should be even ...
- 208
2
votes
1
answer
37
views
Labeling policy for airplane detecting YOLO
I am training my YOLO to detect airplanes and drones. in some of the pictures it is impossible to detect that the object is indeed an airplane, and it even looks like a drone (pictures are taken from ...
- 21
1
vote
1
answer
97
views
Which batch size is optimal for my neural network?
I am currently training a neural network to evaluate chess positions. Everything so far works great, but I am unsure which batch size to choose. I am considering batch sizes of 32, 64, or 128. ...
- 13
0
votes
0
answers
28
views
Why do diffusions learns accumulated noise and not intermediate noise?
The core step is to learn scoring function: $L=\int E_{X_t}\|S_\theta(X_t,t)-\nabla\ln p_{t}(X_t)\|dt$, where $X_t$ is a forward-noise process and $p_{t}(x)$ is a density of X_t.
The trick to actually ...
- 101
0
votes
1
answer
101
views
Matrix Multiplication using a Neural Network
As I understand Neural Networks they have a slow training phase (quadratic or cubic time) and a fast (linear time) inference phase.
Also, the slow training phase comes from the requirement of doing ...
-1
votes
1
answer
60
views
Jetson Orin NX for deep learning training [closed]
I'd like to buy a computer for deep learning experiments.
I found various articles, which proposed a PC system with an NVIDIA RTX 3060 card.
Recently, I discovered Jetson Orin NX Systems, which seem ...
- 99
0
votes
0
answers
40
views
RAG and LangChain for the Generative AI Chatbot to look at only the provided data
Requirements:
It would behaves like ChatGPT, however if user is asking something that is not in the provided data, it will say "Sorry, I do not know such information" <-- Please kindly ...
- 1
0
votes
0
answers
24
views
What's the advantage of multi-GPU training in Alex-Net?
I was reading ImageNet Classification with Deep Convolutional Neural Networks(Alex et al) and they trained their model on two GPUs following fine-grained structure. Can you tell me why they chose that ...
0
votes
1
answer
35
views
Combinig output of two different machine learning models for accurate invoice data extraction: Is this a viable approach?
I am working (trying to work) on a project to extract relevant information from invoices. Currently I don't achieve much good accuracy so am trying to come up with some new ideas. I am considering ...
- 1
0
votes
1
answer
29
views
Regression model training improvement
I am fairly new to TensorFlow and ML in general and am currently working on a regression neural network while learning about different parts and concepts of it. My goal is to try & achieve a model ...
- 101
0
votes
0
answers
57
views
Downsides of training a neural network in constant time
Assumption
Let's assume we collect a high quality amount of training data for machine translation for example parallel corpus data from the european parlament combined with other data. We store these ...
0
votes
0
answers
59
views
why YOLO models multiple the loss by batch size in detection head?
here
return loss.sum() * batch_size, loss.detach() # loss(box, cls, dfl)
This line is from yolov8, but I saw similar thing in v5 too.
So far I only see this kind ...
- 101
1
vote
1
answer
58
views
Are there any machine learning frameworks, other than Statistical Decision Theory which proposes a loss function, for developing models?
In Elements of Statistical Learning, the authors present a framework called statistical decision theory that employs a loss function for choosing $f(X)$ to predict $Y$:
$$EPE(f) = E(L(Y, f(X)))$$
Can ...
user84073
0
votes
0
answers
37
views
NN training by optimizing hidden layer content with information bottleneck "inf_T I(X;T) - beta I(T;Y)"?
There was a lot of hype about Naftali Tishby's information bottleneck method a few years ago, but it is nearly silent now, especially that sadly the author has died in 2021.
In theory it allows to ...
- 101
2
votes
1
answer
100
views
Will an AI LLM learn a language if fed during training with a large corpus of undeciphered language?
AIs can learn many languages just by being trained on their corpora.
What will happen if we in addition would train it on a large corpus of undeciphered language, like Minoan or Etruscan? Will it be ...
- 351
2
votes
1
answer
185
views
How would I train a neural net on multiple different computers?
I have no experience with AI whatsoever, and I'm considering creating a relatively simple model just to actually get into AI creation. However, I have a relatively bad GPU (GT 1030) and I'm worried ...
- 23
0
votes
0
answers
33
views
Advice on Handling Variable Input Sizes in a Neural Network Model for Predicting Football Match Outcomes
I am developing a machine learning model aimed at predicting football match outcomes based on past team performances. The model incorporates data from the last 10 home games for the home team and the ...
- 1
0
votes
1
answer
97
views
How to interpret this training
I'm still learning ml/ai and I'm running a training where the curves look like this.
I was told that this looks good by some and that it doesn't look good by others... But none told me exactly why, I ...
- 103
0
votes
0
answers
20
views
NN with user as a teacher
I have similar question to the following one.
Could an AI be built to learn based of interaction with a human?
What category of NN is the situation when the network learns from users feedback? The NN ...
- 1
0
votes
0
answers
18
views
How to 'compose' a sport video based on .. action
I'm recording some football game using two "fixed" action cam, starting recording both at the same time.
I'm here to ask you the best approach (if exists) to train a program to take each ...
- 101
2
votes
1
answer
146
views
How does Training and Validating work with Graph CNNs
I'm training a Graph Convolutional Neural Network to output embeddings for nodes that I eventually want to perform classification on. I am a little confused on how the training, validation, and ...
- 103
0
votes
0
answers
34
views
Model suggestion for AI based scaling
We are exploring the idea of scaling elements within a UI container based on the given size.
The container is represented by a json object, for example:
...
- 101
0
votes
0
answers
30
views
How to Implement a Text-Based Question Answering System for PDF Documents using Machine Learning?
As a beginner in machine learning, I've completed a basic text classification project in university. Now, I'm eager to build a system that can answer specific questions from a large collection of PDF ...
- 101
0
votes
0
answers
17
views
Can you iteratively freeze and unfreeze parts of a neural network for efficient training?
I know you can do efficient training by freezing parts of a NN, but is there any work done where part 1 of a NN is frozen and part 2 is trained, and then part 2 is frozen and part 1 is trained?
- 233
0
votes
0
answers
31
views
How are conditional models different from supervised models?
I'm wondering what the difference between conditional learning and supervised learning is - especially in diffusion models? Am I correct to assume that diffusion models are supervised because in ...
- 1
0
votes
0
answers
41
views
From how many experts does LLM training using a mixture of experts (MoE) start slowing down compared to training LLMs without MoE?
I'm trying to find some information regarding the impact of the number of experts on LLM training. From how many experts does LLM training using a mixture of experts (MoE) start slowing down compared ...
- 3,267
11
votes
2
answers
6k
views
Can you train a neural network by simply giving it ratings each time it runs?
I am currently trying to train a bot for a game I am creating. It is a 2d game with a complex map made of various shapes. The bot and character shoot bullets that are capable of ricocheting. The ...
- 113
0
votes
0
answers
58
views
Flipping train and test labels for binary classification
I was training a GCN (this one) on a single graph (n=1,1304 nodes, num_features=26) to perform node level binary classification. However, my model performed with 5% accuracy (and even went as low to 0%...
- 1
0
votes
0
answers
24
views
Multimodal architecture: how to train it
I have a set of graphs, a set of sequences and a set of summary stats, the elements in the three sets correspond to each other. I am not satisfied with the regression performance from one single ...
- 101
0
votes
1
answer
93
views
model.fit fails using keras sequential (slice index ### of dimension 0 out of bounds) [closed]
this is the most simple model I can think of for my data yet I can't use the fit function, it gives an error.
the desired procedure is to make a simple autoencoder : from 576 nodes to 64 then back to ...
- 23
2
votes
1
answer
104
views
What are these special "AI chips" actually used for?
To make an AI / LLM, like ChatGPT, you need two things:
To create the LLM. This includes training it, etc. Very expensive from computation perspective.
Run the LLM to answer user queries. For ...
- 121
0
votes
0
answers
57
views
Can we use quantization techniques to train ML models from scratch?
I'm currently learning about quantization and have seen that most methods for quantizing are for a Post-Training conversion or, in some cases, for efficient finetuning like the QLORA method. I'd like ...
- 660
0
votes
2
answers
86
views
Why are hallucinations a huge problem?
Why it is not easy to just train model for factual accuracy and punish for any invented facts or mistakes, award for admitting ignorance in difficult cases?
- 351