I have a scenario in which we should leverage previously asked questions (not questions pairs, single question in a column) to locate similar questions within those questions.

How can I fine-tune my model to manage out of vocabulary, as my data includes domain-specific questions (3300 questions)?.

Right now, I'm using hugging face sentence transformers, which is already pre-trained on huge data.

For example, BERT knows that gold is a metal, but, in our domain corpus, it's a platform. We have some terminologies which were not exposed openly, how can I fine-tune the model to get related sentences (handling OOV).

  • $\begingroup$ Hello. I'm not sure I understand this question/problem. Let me try to understand if I understood your problem. You have some pre-trained model. You also have your own dataset. However, your dataset contains terms, which are equal to the terms used to pre-train the model, but they have different meaning, but you want the pre-trained model to learn the meaning of the words in your corpus and forget the meaning it's learned before. Is this the question/problem? Btw, please, put your specific question in the title. Right now, it's very vague/general. $\endgroup$
    – nbro
    Sep 15, 2021 at 1:18
  • $\begingroup$ your understanding is correct, lets say the question : How to access gold environment? (here gold is an technical word in our organization). But, in general gold is a metal. If I would search gold then more importance should be for gold(environment) and not for gold(metal). $\endgroup$
    – smanem
    Sep 15, 2021 at 5:30
  • $\begingroup$ Ok, but, in your case, the model is supposed to produce whole sentences and not words, it seems from your description. So, rather than the meaning of words, which can differ in your corpus, you want to fine-tune this model so that it understands that your sentences (in your corpus), which are equal to the sentences used in the corpus used to pre-train the model, have different meaning that the other equal sentences. Or maybe the sentences are different and you're just concerned with the words in the sentences? $\endgroup$
    – nbro
    Sep 15, 2021 at 12:55
  • $\begingroup$ See my last edit to your post and, in particular, the title, and make sure it's consistent with your actual question. $\endgroup$
    – nbro
    Sep 15, 2021 at 12:55


You must log in to answer this question.

Browse other questions tagged .