New answers tagged natural-language-processing
1
The 'vocabulary' used is:
$\mathscr{F}$ - fluents, conditions that change over time. The predicate holds(<fluent>,<state>) defines whether a condition applies in a given state.
$\mathscr{A}$ - actions, describe, erm, actions that happen. The predicate occurs(<action>,<state>) specifies that a given action happens at a particular ...
2
The most distinct words of a language are usually the function words (the, and, of, with,...); other lexical items are often (at least partly) shared between languages that had come in contact with each other. So looking for function words is usually the best way to identify the language in a given text.
This can be done by having a list of function words ...
0
According to a nice article by Sebastian Ruder https://ruder.io/4-biggest-open-problems-in-nlp/ based on answers from top NLP researchers https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit
Natural language understanding
NLP for low-resource scenarios
Reasoning about large or multiple documents
Datasets, problems, and ...
1
Answer to Q1) If sampling for next token do you need to apply mask during inference?
Yes you do! The models ability to transfer information across positions was trained in this manner, and changing it up will have unpredictable consequences. Let my try to give an example:
Tokens: 1:sally, 2:sold, 3:seashells, 4:on, 5:the, 6:____
In the above you are ...
Top 50 recent answers are included
