2 votes
Accepted

Fine Tuning a Bert Transformer. How to label for emotions and train large scripts?

From my understanding of your task, you're looking to get the overall emotion classification score for a long piece of dialogue. BERT can handle contexts up to 512 tokens in length, so the task ...
Alexander Wan's user avatar
1 vote
Accepted

Keywords extractions from short names (table and column names)

As it seems like each keyword column is relatively short, I think the best approach here would be to try every possible phrase (i.e., every consecutive span on ...
Alexander Wan's user avatar
1 vote
Accepted

How to label missing/default values for a named entity recognition dataset

NER models are usually trained on non overlapping sequence of tokens. And this is pretty much the only rule followed (even though it's not strictly required to do so, but at the risk of complicating ...
Edoardo Guerriero's user avatar
1 vote

How word2vec de-embeds the special names in language models which output text

I'm not sure I got 100% of the question, but word2vec is rather simple to understand so I'll give my two cents and if something is missing or not clear I'll edit the answer to improve it. 1 For OOV ...
Edoardo Guerriero's user avatar
1 vote

Why do the values in the cross attentional mechanism within a transformer come from the encoder and not from the decoder?

The idea of the cross-attention layer is to transform the input words to output words. The Decoder provides context of which input words should we pay attention to next based on the already decoded ...
Abhishek's user avatar
  • 111

Only top scored, non community-wiki answers of a minimum length are eligible