Skip to main content

Questions tagged [huggingface]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers

QLora using peft from HF and custom class for binary classification

I am fine-tuning an mistral-7B LLM model for binary classification. I realize it may be an overkill; but we are running some experiments. So far, I have used HuggingFace libraries like peft and ...
kms's user avatar
  • 109
0 votes
1 answer

Train my own LLM on a smaller corpus of text?

Would it be possible to train my own LLM on a smaller corpus of text, lets say some coding documentation that I then want to ask questions about using the model? If so, are there any recommended ways ...
Dylan Dijk's user avatar
-1 votes
1 answer

Why are some of the weights not initialized from the pretrained model checkpoint (from hugging face)? [closed]

Sebastian Nielsen's user avatar
0 votes
1 answer

My small BERT can't even overfit on a sentiment analysis task

I'm trying to train (from scratch) a miniature BERT model on SST2, a simple binary sentiment analysis task with inputs of maybe 5-20 words at a time. As you can see in my code, my approach is a little ...
Jack M's user avatar
  • 262