Questions tagged [pretrained-models]
For questions related to pre-trained model. A pre-trained model is a model that was trained on a large benchmark dataset to solve a problem similar to the one that we want to solve. Accordingly, due to the computational cost of training such models, it is common practice to import and use models from published literature (e.g. VGG, Inception, MobileNet)
23
questions
1
vote
1
answer
36
views
What is the difference between fine tuning and variants of few shot learning? [duplicate]
I am trying to understand the concept of fine-tuning and few-shot learning.
I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task.
However, ...
- 51
0
votes
0
answers
32
views
Modify the architecture of the pre-trained model
I'm trying to remove some layers and reduce the size of fully connected in pre-trained BERT model.
And then I'm going to fine-tune the modified models on GLUE. But the problem is that the modified ...
- 1
0
votes
0
answers
33
views
Algorithm to separate audio source that Andrew Ng did in his ML Course
Hey I was doing the ML course of Andrew Ng's ML course. In a video he shows a algorithm that can separate two audio source and separate voice from music. Has any one tried such an algorithm? Though he ...
0
votes
1
answer
249
views
Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?
I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
- 103
0
votes
1
answer
724
views
Does BERT freeze the entire model body when it does fine-tuning?
Recently, I came across the BERT model. I did some research and tried some implementations.
I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace.
...
- 1
0
votes
2
answers
234
views
How to design a neural network with arbitrary input and output length?
I am trying to build a neural network that has an input of $n$ pairs of integer values (where $n$ is random) and a corresponding output of a binary array with length $n$.
The input will be a set of ...
- 1
0
votes
1
answer
74
views
How to perform multi-class text classification with a dataset of 80 documents?
I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags.
How can I perform multi-class text classification with such a small ...
-1
votes
1
answer
54
views
How to train my model using transfer learning on inception_v3 pre-trained model?
I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy.
My dataset is from kaggle:
https://www....
2
votes
1
answer
71
views
What are some most promising ways to approximate common sense and background knowledge?
I learned from this blog post Self-Supervised Learning: The Dark Matter of Intelligence that
We believe that self-supervised learning (SSL) is one of the most promising ways to build such background ...
- 741
1
vote
0
answers
30
views
How to scrape product data on supplier websites?
I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
- 11
0
votes
1
answer
377
views
How to use a conv2d layer after a flatten?
I am not familiar with Deep learning and Pytorch. And I want to know how to deal, in general with such a situation. So, I was wondering if I used a pretrained model (EfficientNet for example) if I ...
- 35
0
votes
0
answers
34
views
Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?
I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
- 101
2
votes
1
answer
110
views
Which hyperparameters in neural network are accesible to users adjustment
I am new to Neural Networks and my questions are still very basic.
I know that most of neural networks allow and even ask user to chose hyper-parameters like:
amount of hidden layers
amount of ...
- 131
0
votes
1
answer
50
views
BERT: After pretraining 880000 step, why fine-tune not work? [closed]
I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples
Pretrain parameters:
...
- 101
1
vote
0
answers
80
views
Is it good practice to save NLP Transformer based pre-trained models into file system in production environment
I have developed a multi label classifier using BERT. I'm leveraging Hugging Face Pytorch implementation for transformers.
I have saved the pretrained model into the file directory in dev environment. ...
- 141
1
vote
0
answers
48
views
How to measure/estimate the energy consumption of CNN models during testing?
Does someone know a method to estimate / measure the total energy consumption during the test phase of the well-known CNN models? So with a tool or a power meter...
MIT has already a tool to estimate ...
- 11
0
votes
0
answers
129
views
Can StyleGAN be refined without a full training?
Can I refine StyleGAN or StyleGAN2 without retraining it for many days, such that its pretrained model is trained to generate only faces similar to a (rather small) set of reference images?
I would ...
- 240
0
votes
1
answer
116
views
How to add a pretrained model to my layers to get embeddings?
I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the ...
- 41
1
vote
1
answer
5k
views
How to use pre-trained BERT to extract the vectors from sentences?
I'm trying to extract the vectors from the sentences. Spent soo much time searching for the pre-trained BERT models but found nothing.
Is it possible to get the vectors using pre-trained BERT from ...
- 1,193
3
votes
1
answer
319
views
How does a software license apply to pretrained models?
Google provides a lot of pretrained tensorflow models, but I cannot find a license.
I am interested in the tfjs-models. The code is licensed Apache-2.0, but the models are downloaded by the code, so ...
- 240
1
vote
0
answers
42
views
Can mobilenet in some cases perform better than inception_v3 and inception_resnet_v2?
I have implemented a multi-label image classification model where I can choose which model to use, I was surprised to find out that in my case mobilenet_v1_224 performed much better (95% Accuracy) ...
- 125
4
votes
0
answers
43
views
How can I improve the performance of a model trained to detect vehicle poses?
I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain ...
- 41
3
votes
1
answer
247
views
Are there any better visual models for transfer rather than ImageNet?
Similar to the recent pushes in Pretrained Language Models (BERT, GPT2, XLNet) I was wondering if such a thrust exists in Computer Vision?
From my understanding, it seems the community has converged ...
- 2,249