New answers tagged natural-language-processing
1
Let's start with RNN. A well known problem is vanishin/exploding gradients, which means that the model is biased by most recent inputs in the sequence, or in other words, older inputs have practically no effect in the output at the current step.
LSTMs/GRUs mainly try to solve this problem, by including a separate memory (cell) and/or extra gates to learn ...
1
I'll list some bullet points of the main innovations introduced by transformers , followed by bullet points of the main characteristics of the other architectures you mentioned, so we can then compared them.
Transformers
Transformes (Attention is all you need) were introduced in the context of machine translation with the purpose to avoid recursion in ...
0
If you look at the words in your dictionary (vocab) before/after pruning, most likely you'd see there isn't a lot of difference, not so much to affect your model performance.
In fact, creating a dictionary and model training are two more or less indpendent processes. To make your life easier, you could find the largest dev set you can find for building ...
1
Each node is a position in the arrays
values = value of the node
conn = indexes of connected nodes
If its an undirected graph, each node must have all the nodes to which they are attached.
Instead, in directed graphs, only the start node has the index.
For your image:
values = ['A','B','C','D','E']
conn = [[1,2,3],[0,4],[0,3,4],[0,2,3],[1,3]]
Example = '...
1
All depends on quality of data. Due to old rule Garbage in, garbage out
https://en.wikipedia.org/wiki/Garbage_in,_garbage_out, if you have bad quality data(data redundancy, unstructured data, to much memory etc) your results won't be spectacular. In other cases, everybody could be a Data Scientist, because its only task was "put raw text into classifier". ...
0
Supervised
Pros:
highest accuracy
Cons:
need a large human-labeled training set
brittle (doesn't work well with examples that are in a different genre from the training set)
Semi-supervised
Relation bootstrapping
Pros:
only requires a small set of labeled data (seed relations)
Cons:
complex iterative process
Distant supervision
Pros:
training ...
2
I have not found any simple implementation of a naive EBMT system, but I found some articles, papers and books that may be helpful (although I haven't read them, apart from the first and last one), so I will list them below.
The web article Example-based machine translation provides a decent high-level explanation of example-based machine translation.
The ...
3
I would say that the logic behind the introduction was more empirical than technical. The only difference between LSTM and Bi-LSTM is the possibility for Bi-LSTM to leverage future context chunks to learn better representations of single words. There is no special training step or units added, the idea is just to read a sentence forward and backward to ...
0
Speech recognition and Character recognition are not part of NLP. Everything else on your list can in principle be done with RNNs. But the field is quickly moving towards using transformers.
1
The blog post Unsupervised Cross-lingual Representation Learning (2019), the related paper and slides by Sebastian Ruder (a researcher currently at DeepMind) summarize what you are looking for. In fact, the authors write
We will introduce researchers to state-of-the-art methods for constructing resource-light cross-lingual word representations and discuss ...
3
Dialogue is a hard problem because it requires pretty advanced cognitive functions. Leaving aside all the lower levels of language analysis (phonology if dealing with speech, morphology and syntax), you quickly run into interpretation problems that require a lot of world knowledge.
Simple question and answer is fine, and restricted domains are somewhat ...
0
According to Dan Jurafsky, a researcher on NLP and NLU, the current hard problems in NLP are (see slide 6)
Questioning answering
Paraphrase
Summarisation
Dialogue
Other hard problems for which there are already some good solutions are
Sentiment analysis
Coreference resolution
Word sense disambiguation
Parsing
Machine translation
Information extraction
...
1
First of all, I am not very familiar with details of NLP and NLU systems and concepts, so I will provide an answer based on the slides entitled Natural language understanding in dialogue systems (2013) by David DeVaul, a researcher on the topic.
A dialogue system is composed of different parts or modules. Here's a diagram of an example of a dialogue system.
...
2
So as you're probably already aware of, CBOW and Skip-gram are just mirrored versions of each other. CBOW is trained to predict a single word from a fixed window size of context words, whereas Skip-gram does the opposite, and tries to predict several context words from a single input word.
Intuitively, the first task is much simpler, this implies a much ...
0
I don't think this classify as an NLP problem, there is almost no semantic analysis needed, it is more like a classification problem using categorical features.
NLTK is surely valuable if you want to perform some text 'cleaning' or preprocessing before encoding the variables. The only NLP application that I think you could apply here is some sentiment ...
0
There is a general idea in the field of NLP that there is a mapping between embeddings in different langauges. Figure 1 explains this.
In Figure 1. we have the embedding of English words and Spanish words, and we see that their exists a mapping between the manifolds associated to this two languages, i.e. Spanish manifold is a distorted image of the English ...
1
Embeddings generated by transformers like Bert or XLM-R are fundamentally different from embeddings learned through language models like GloVe or Word2Vec.
The latter are static, i.e. they are just dictionaries containing a vocabulary with n-dimensional vectors associated to each word. Because of this they can be plotted through PCA and the distance between ...
1
Have a look at the paper A Modular Architecture for Unsupervised Sarcasm Generation (2019) by Mishra et al.
In the abstract, the authors write
In this paper, we propose a novel framework for sarcasm generation; the system takes a literal negative opinion as input and translates it into a sarcastic version. Our framework does not require any paired data ...
Top 50 recent answers are included
