New answers tagged natural-language-processing
1
Although asked over 3 years ago, the question is still interesting and while I agree with the original answer, a lot can be added to it.
First, I'd like to point out that the term "knowledge base" is very ambiguous and it means different things to different people. For example, there is no sharp distinction between knowledge base and neural network....
1
The following figure from this article can be helpful:
This figure represents "Skip-Gram model structure. Current center word is 'passes'".
2
We give the target input into the transformer decoder while training the model. So it is easy for the model to "peek ahead" and learn what the next word would be. To ensure that this doesn't happen we apply an additive mask after the dot product between Query and Key. In the original paper "Attention is all you need", the triangular ...
3
The Transformer model presented in this tutorial is an auto-regressive Transformer. Which means that prediction of next token only depends on it's previous tokens.
So in order to predict next token, you have to make sure that only previous token are attended. (If not, this would be a cheating because model already knows whats next).
So attention mask would ...
0
I think it's difficult to tell wich algorithm is "the best" or "the simplest".
I had the same issue of choosing the suited NLP algorithm for my dataset and I used :
https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html
Then I recommanded you to test many algorithms as you can to find the best for your needed.
2
If you go through the main introductory paper of the transformer ("Attention is all you need"), you can find the comparison of the model with other state-of-the-art machine translation method:
For example, Deep-Att + PosUnk is a method that has utilized RNN and attention for the translation task. As you can see, the training cost for the ...
1
The problem you state is a well known problem, and it is called "keyword spotting" os KWS. If you add a wake up word before it (like "hey google/siri"), you can also use "voice command" system to alleviate the problem.
There are two kind of KWS systems: those which develop to detect a hard coded set of keywords, and those who ...
0
Thank you very much for your help, all of you.
I finally find on the Internet key words : "Dialog act classification".
I don't know yet how to implement it, but it's a good start !
1
This is a difficult problem.
First, how do you define 'subject'? Do you have a (closed) lists of labels you want to assign? What about subjects that overlap, or don't occur in your list? What even is a subject? This is a non-trivial issue.
Second, and this is even harder, how do you want to recognise subjects? A simple solution could be using a list of ...
4
I will be starting my PhD in natural language processing in a few days and this is very similar to my proposed topic. It's an open problem that ties NLP and AI into philosophy of science and epistemology and is, I think, extremely interesting. I say all this to drive home the point that this is not a simple problem.
Two major theoretical concerns come to my ...
0
What is a transformer?
The original transformer, proposed in the paper Attention is all you need (2017), is an encoder-decoder-based neural network that is mainly characterized by the use of the so-called attention (i.e. a mechanism that determines the importance of words to other words in a sentence or which words are more likely to come together) and the ...
Top 50 recent answers are included
