Questions tagged [pretrained-models]

For questions related to pre-trained model. A pre-trained model is a model that was trained on a large benchmark dataset to solve a problem similar to the one that we want to solve. Accordingly, due to the computational cost of training such models, it is common practice to import and use models from published literature (e.g. VGG, Inception, MobileNet)

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

TD3 Retraining from simple to complex environment

I have a robotic environment for cable robots in Unity. I am training a TD3 agent for a robot reaching task where cable lengths are the control inputs and [current cables, robot position, target ...
Rohit's user avatar
  • 1
0 votes
0 answers
22 views

Question about the implications of the AI Act

the EU is planning to release the AI act which will regulate all AI-Systems that are made available in the Europe. Recently, a new draft was published, containing the following passage: ... providers ...
al-canonic's user avatar
0 votes
0 answers
12 views

Looking for an audio classification approach

I am working on a deepfake audio classification project with a dataset consisting of only 3000 samples. I have made several attempts to address this challenge. Firstly, I extracted melspectrograms and ...
user21456801's user avatar
0 votes
1 answer
34 views

Is size of trained model on disk a good measure of model complexity?

I am writing a research paper on my own custom CNN model for image classification. I am comparing my model architecture with pre-trained architectures, like DenseNet121 and InceptionV3. I want to ...
Dawood Ahmad's user avatar
0 votes
0 answers
62 views

Can pretraining be continued after RLHF?

Assume you have a pretrained transformer language model (M1) which already underwent reinforcement learning by human feedback (M2). I assume that it is in principle possible to continue the ...
Hans-Peter Stricker's user avatar
0 votes
0 answers
19 views

Normalisation in feature extraction using pre-trained model

I have a dataset with medical images. I want to implement a network for super-resolution using GANs. One of the criteria of the optimisation is a perceptual loss. For that I will use a pretrained vgg ...
Janikas's user avatar
  • 101
2 votes
1 answer
243 views

Should I use pretrained model for image classification or not?

I have thousands of images similar to this. I can classify them using existing metadata to different folders according to gravel product type loaded on the truck. What would be optimal way to train a ...
Vojtěch Dohnal's user avatar
0 votes
1 answer
29 views

Do different ngrams share embedding in Fasttext?

As per Section 3.2 in the original paper on Fasttext, the authors state: In order to bound the memory requirements of our model, we use a hashing function that maps n-grams to integers in 1 to K ...
Fijoy Vadakkumpadan's user avatar
0 votes
0 answers
68 views

Help on Object-Detection Task - Detection of Cracks on Walls (Pre-Trained Models etc.)

I'm member of a University Project Team in D.U.Th., a university in Greece. Lately, we have been trying to implement a Neural Network Model for our Project and, so far, we have had some progress worth ...
Eliasmys's user avatar
0 votes
0 answers
38 views

How to tokenize compound sentences based on the conjunctions?

I am trying to tokenize sentences of a document for aspect-based sentiment analysis. There are some sentences that consist of more than one topic. For example, The touch screen is good but the ...
mansoor sh's user avatar
0 votes
1 answer
392 views

Fine Tuning Transformer Model for Machine Translation

I am working on the Transformer example demonstrated on TensorFlow's website. https://www.tensorflow.org/text/tutorials/transformer In this example, Machine Translation model is trained to translate ...
boyaronur's user avatar
  • 101
3 votes
0 answers
816 views

What is the difference between prompt tuning and prefix tuning?

I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really? ...
Exploring's user avatar
  • 303
3 votes
1 answer
265 views

Using a pre-trained model to generate labels to data to then train a model on

I'm trying to set up a pipeline for my ML models to automatically re-train themselves whenever concept drift occurs to recalibrate to the new output distributions. However, I can't get ground-truth ...
Sanger Steel's user avatar
1 vote
1 answer
355 views

How to Train a Decoder for Pre-trained BERT Transformer-Encoder?

Context: I am currently working on an encoder-decoder sequence to sequence model that uses a sequence of word embeddings as input and output, and then reduces the dimensionality of the word embeddings....
node_env's user avatar
4 votes
1 answer
4k views

What is the difference between fine tuning and variants of few shot learning? [duplicate]

I am trying to understand the concept of fine-tuning and few-shot learning. I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task. However, ...
Exploring's user avatar
  • 303
0 votes
1 answer
1k views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
Araw's user avatar
  • 103
2 votes
1 answer
3k views

Does BERT freeze the entire model body when it does fine-tuning?

Recently, I came across the BERT model. I did some research and tried some implementations. I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace. ...
Joon's user avatar
  • 51
0 votes
2 answers
890 views

How to design a neural network with arbitrary input and output length?

I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$. The input will be a set of ...
Kian's user avatar
  • 1
0 votes
1 answer
122 views

How to perform multi-class text classification with a dataset of 80 documents?

I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags. How can I perform multi-class text classification with such a small ...
Utkarsh Malkoti's user avatar
-1 votes
1 answer
136 views

How to train my model using transfer learning on inception_v3 pre-trained model?

I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy. My dataset is from kaggle: https://www....
Shubham Agrawal's user avatar
2 votes
1 answer
98 views

What are some most promising ways to approximate common sense and background knowledge?

I learned from this blog post Self-Supervised Learning: The Dark Matter of Intelligence that We believe that self-supervised learning (SSL) is one of the most promising ways to build such background ...
Lerner Zhang's user avatar
1 vote
0 answers
58 views

How to scrape product data on supplier websites?

I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
johannesha's user avatar
0 votes
1 answer
794 views

How to use a conv2d layer after a flatten?

I am not familiar with Deep learning and Pytorch. And I want to know how to deal, in general with such a situation. So, I was wondering if I used a pretrained model (EfficientNet for example) if I ...
JoJolyne's user avatar
0 votes
0 answers
85 views

Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?

I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
yusuf's user avatar
  • 101
2 votes
1 answer
135 views

Which hyperparameters in neural network are accesible to users adjustment

I am new to Neural Networks and my questions are still very basic. I know that most of neural networks allow and even ask user to chose hyper-parameters like: amount of hidden layers amount of ...
Igor's user avatar
  • 181
0 votes
1 answer
64 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
惊天补扣's user avatar
1 vote
0 answers
116 views

Is it good practice to save NLP Transformer based pre-trained models into file system in production environment

I have developed a multi label classifier using BERT. I'm leveraging Hugging Face Pytorch implementation for transformers. I have saved the pretrained model into the file directory in dev environment. ...
Murugesh's user avatar
  • 141
1 vote
0 answers
80 views

How to measure/estimate the energy consumption of CNN models during testing?

Does someone know a method to estimate / measure the total energy consumption during the test phase of the well-known CNN models? So with a tool or a power meter... MIT has already a tool to estimate ...
Mugiwara San's user avatar
0 votes
0 answers
147 views

Can StyleGAN be refined without a full training?

Can I refine StyleGAN or StyleGAN2 without retraining it for many days, such that its pretrained model is trained to generate only faces similar to a (rather small) set of reference images? I would ...
allo's user avatar
  • 300
0 votes
1 answer
202 views

How to add a pretrained model to my layers to get embeddings?

I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the ...
inquisitive's user avatar
2 votes
1 answer
6k views

How to use pre-trained BERT to extract the vectors from sentences?

I'm trying to extract the vectors from the sentences. Spent soo much time searching for the pre-trained BERT models but found nothing. Is it possible to get the vectors using pre-trained BERT from ...
Pluviophile's user avatar
  • 1,263
3 votes
1 answer
639 views

How does a software license apply to pretrained models?

Google provides a lot of pretrained tensorflow models, but I cannot find a license. I am interested in the tfjs-models. The code is licensed Apache-2.0, but the models are downloaded by the code, so ...
allo's user avatar
  • 300
1 vote
0 answers
71 views

Can mobilenet in some cases perform better than inception_v3 and inception_resnet_v2?

I have implemented a multi-label image classification model where I can choose which model to use, I was surprised to find out that in my case mobilenet_v1_224 performed much better (95% Accuracy) ...
ATidedHumour's user avatar
4 votes
0 answers
51 views

How can I improve the performance of a model trained to detect vehicle poses?

I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain ...
mechane's user avatar
  • 41
3 votes
1 answer
642 views

Are there any better visual models for transfer rather than ImageNet?

Similar to the recent pushes in Pretrained Language Models (BERT, GPT2, XLNet) I was wondering if such a thrust exists in Computer Vision? From my understanding, it seems the community has converged ...
mshlis's user avatar
  • 2,359