Questions tagged [pretrained-models]

For questions related to pre-trained model. A pre-trained model is a model that was trained on a large benchmark dataset to solve a problem similar to the one that we want to solve. Accordingly, due to the computational cost of training such models, it is common practice to import and use models from published literature (e.g. VGG, Inception, MobileNet)

Filter by
Sorted by
Tagged with
1 vote
1 answer
36 views

What is the difference between fine tuning and variants of few shot learning? [duplicate]

I am trying to understand the concept of fine-tuning and few-shot learning. I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task. However, ...
user avatar
0 votes
0 answers
32 views

Modify the architecture of the pre-trained model

I'm trying to remove some layers and reduce the size of fully connected in pre-trained BERT model. And then I'm going to fine-tune the modified models on GLUE. But the problem is that the modified ...
user avatar
  • 1
0 votes
0 answers
33 views

Algorithm to separate audio source that Andrew Ng did in his ML Course

Hey I was doing the ML course of Andrew Ng's ML course. In a video he shows a algorithm that can separate two audio source and separate voice from music. Has any one tried such an algorithm? Though he ...
user avatar
0 votes
1 answer
249 views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
user avatar
  • 103
0 votes
1 answer
724 views

Does BERT freeze the entire model body when it does fine-tuning?

Recently, I came across the BERT model. I did some research and tried some implementations. I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace. ...
user avatar
  • 1
0 votes
2 answers
234 views

How to design a neural network with arbitrary input and output length?

I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$. The input will be a set of ...
user avatar
  • 1
0 votes
1 answer
74 views

How to perform multi-class text classification with a dataset of 80 documents?

I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags. How can I perform multi-class text classification with such a small ...
user avatar
-1 votes
1 answer
54 views

How to train my model using transfer learning on inception_v3 pre-trained model?

I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy. My dataset is from kaggle: https://www....
user avatar
2 votes
1 answer
71 views

What are some most promising ways to approximate common sense and background knowledge?

I learned from this blog post Self-Supervised Learning: The Dark Matter of Intelligence that We believe that self-supervised learning (SSL) is one of the most promising ways to build such background ...
user avatar
1 vote
0 answers
30 views

How to scrape product data on supplier websites?

I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
user avatar
0 votes
1 answer
377 views

How to use a conv2d layer after a flatten?

I am not familiar with Deep learning and Pytorch. And I want to know how to deal, in general with such a situation. So, I was wondering if I used a pretrained model (EfficientNet for example) if I ...
user avatar
0 votes
0 answers
34 views

Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?

I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
user avatar
  • 101
2 votes
1 answer
110 views

Which hyperparameters in neural network are accesible to users adjustment

I am new to Neural Networks and my questions are still very basic. I know that most of neural networks allow and even ask user to chose hyper-parameters like: amount of hidden layers amount of ...
user avatar
  • 131
0 votes
1 answer
50 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
user avatar
  • 101
1 vote
0 answers
80 views

Is it good practice to save NLP Transformer based pre-trained models into file system in production environment

I have developed a multi label classifier using BERT. I'm leveraging Hugging Face Pytorch implementation for transformers. I have saved the pretrained model into the file directory in dev environment. ...
user avatar
  • 141
1 vote
0 answers
48 views

How to measure/estimate the energy consumption of CNN models during testing?

Does someone know a method to estimate / measure the total energy consumption during the test phase of the well-known CNN models? So with a tool or a power meter... MIT has already a tool to estimate ...
user avatar
0 votes
0 answers
129 views

Can StyleGAN be refined without a full training?

Can I refine StyleGAN or StyleGAN2 without retraining it for many days, such that its pretrained model is trained to generate only faces similar to a (rather small) set of reference images? I would ...
user avatar
  • 240
0 votes
1 answer
116 views

How to add a pretrained model to my layers to get embeddings?

I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the ...
user avatar
1 vote
1 answer
5k views

How to use pre-trained BERT to extract the vectors from sentences?

I'm trying to extract the vectors from the sentences. Spent soo much time searching for the pre-trained BERT models but found nothing. Is it possible to get the vectors using pre-trained BERT from ...
user avatar
  • 1,193
3 votes
1 answer
319 views

How does a software license apply to pretrained models?

Google provides a lot of pretrained tensorflow models, but I cannot find a license. I am interested in the tfjs-models. The code is licensed Apache-2.0, but the models are downloaded by the code, so ...
user avatar
  • 240
1 vote
0 answers
42 views

Can mobilenet in some cases perform better than inception_v3 and inception_resnet_v2?

I have implemented a multi-label image classification model where I can choose which model to use, I was surprised to find out that in my case mobilenet_v1_224 performed much better (95% Accuracy) ...
user avatar
4 votes
0 answers
43 views

How can I improve the performance of a model trained to detect vehicle poses?

I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain ...
user avatar
  • 41
3 votes
1 answer
247 views

Are there any better visual models for transfer rather than ImageNet?

Similar to the recent pushes in Pretrained Language Models (BERT, GPT2, XLNet) I was wondering if such a thrust exists in Computer Vision? From my understanding, it seems the community has converged ...
user avatar
  • 2,249