2
votes
Accepted
Fine Tuning a Bert Transformer. How to label for emotions and train large scripts?
From my understanding of your task, you're looking to get the overall emotion classification score for a long piece of dialogue.
BERT can handle contexts up to 512 tokens in length, so the task ...
- 644
1
vote
Accepted
Keywords extractions from short names (table and column names)
As it seems like each keyword column is relatively short, I think the best approach here would be to try every possible phrase (i.e., every consecutive span on ...
- 644
1
vote
Accepted
How to label missing/default values for a named entity recognition dataset
NER models are usually trained on non overlapping sequence of tokens. And this is pretty much the only rule followed (even though it's not strictly required to do so, but at the risk of complicating ...
- 5,183
1
vote
How word2vec de-embeds the special names in language models which output text
I'm not sure I got 100% of the question, but word2vec is rather simple to understand so I'll give my two cents and if something is missing or not clear I'll edit the answer to improve it.
1 For OOV ...
- 5,183
1
vote
Why do the values in the cross attentional mechanism within a transformer come from the encoder and not from the decoder?
The idea of the cross-attention layer is to transform the input words to output words. The Decoder provides context of which input words should we pay attention to next based on the already decoded ...
- 111
Only top scored, non community-wiki answers of a minimum length are eligible