Questions tagged [natural-language-processing]

For questions related to natural language processing, which concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
1
vote
0answers
14 views

Why hasn't deep learning been used for word level alignment?

I've been exploring word-level alignments tools such as MGIZA and it seems to me that there hasn't been any new tool for this problem. Are neural networks not suitable to solve this problem or simply ...
1
vote
2answers
30 views

Does it matter if it's a bot or a human generating text? Doesn't it come down to the content?

It was noted today that automated text generation is advancing at a rapid pace, potentially accelerating. As bots become more and more capable of passing turing tests, especially in single iterations,...
1
vote
0answers
16 views

How to train a transformer text to text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
1
vote
2answers
42 views

Why is embedding important in NLP, and how does autoencoder work?

People say embedding is necessary in NLP because if using just the word indices, the efficiency is not high as similar words are supposed to be related to each other. However, I still don't truly get ...
0
votes
1answer
19 views

How to change this RNN text classification code to become text generation code?

I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T ...
2
votes
0answers
19 views

What is the State-of-the-Art open source Voice Cloning tool right now?

I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning),...
2
votes
0answers
16 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
1answer
43 views

Why do you need to retrain GPT-2?

I'm following this tutorial, and I wonder why is there a train-step - why is it necessary? I thought the whole idea of GPT-2 is that you do not need to train it on specific text domain, as it's ...
6
votes
2answers
198 views

How would an AI learn idiomatic phrases in a natural language?

After an AI goes through the process described in How would an AI learn language?, an AI knows the grammar of a language through the process of grammar induction. They can speak the language, but they ...
-1
votes
1answer
118 views

Can a neural net learn to read?

I am a student of last year of computer engineering and lately I have been very interested in AI. Fields such as ML and DL seem very disruptive to me. A few months ago I saw an interview of Bill ...
3
votes
1answer
52 views

Techniques for sentence transformation and consistency

I'm looking for NLP techniques to transform sentences without affecting their meaning, and measures of textual consistency. For example, techniques that could transform active voice into passive voice,...
2
votes
2answers
57 views

Chatbot using Machine Learning vs Programming

I am currently building a chatbot. What I have done so far is, collected possible questions/training data/files and created a model out of it using Apache OpenNLP; the model is able to predict all the ...
2
votes
1answer
51 views

How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?

This is related to my earlier question, which I'm trying to break down into parts (this being the first). I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. ...
0
votes
0answers
19 views

Understanding how continuous bag of words method learns embedded representations

I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding: If we have a vocabulary $V$, the naive way to ...
0
votes
1answer
37 views

Preparation of input data

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different ...
2
votes
2answers
36 views

Can ELMO embeddings be used to find the n most similar sentences?

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a ...
0
votes
1answer
30 views

How to use pretrained checkpoints of BERT model on semantic text similarity task?

I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity. ...
1
vote
2answers
40 views

What is “Word Sense Disambiguation”?

I recently came across this article which cites a paper which apparently won outstanding paper in ACL 2019. The theme is that it solved a longstanding problem called Word Sense Disambiguation. What ...
2
votes
0answers
39 views

Grouped Text classification

I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph ...
1
vote
3answers
43 views

Why do language models place less importance on punctuation?

I have very outdated idea about how NLP tasks are carried out by normal RNN's, LSTM's/GRU's, word2vec, etc to basically generate some hidden form of the sentence understood by the machine. One of the ...
1
vote
0answers
6 views

How can we use Dependency Parsers for Negation detection

I am building a negation detection system. How to use dependency parsers for the same. I am using SPACY for dependency parser
0
votes
0answers
23 views

How do I do further (domain specific) pre-training with Google BERT in preparation for subsequent fine-tuning?

Another way to say this is "can you create a .ckpt file created from the final output of BERT?" Here's a recent paper that talks about additional fine-tuning.
1
vote
0answers
17 views

Language Model from missing data

I want to learn how a set of operations (my vocabulary) are composed in a dataset of algorithms (corpus). The algorithms are a sequence of higher level operations which have varying low-level ...
1
vote
0answers
48 views

Machine learning methods to identify the recipient of a document?

I need some advice on what AI methods would be suited to the identification of a recipient of a document, where the format of the documents may vary.
0
votes
0answers
20 views

How can I do entity recognition given the features?

I constructed the features from this tutorial: https://sklearn-crfsuite.readthedocs.io/en/latest/tutorial.html#features. But I did not understand the process of making the model. After doing the ...
3
votes
0answers
22 views

Can computers recognise “grouping” from voice tonality?

In human communication, tonality or tonal language play many complex information, including emotions and motives. But excluding such complex aspects, tonality serves some a very basic purpose of "...
2
votes
0answers
62 views

What is a conditional random field?

I new in machine learning, especially in Conditional Random Fields (CRF). I have read several articles and papers and in there is always associated with HMM and sequences classification. I don't ...
2
votes
0answers
28 views

Models to extract Causal Relationship between entities in a document using Natural Language Processing techniques

I am looking to extract causal relations between entities like Drug and Adverse Effect in a document. Are there any proven NLP or AI techniques to handle the same. Also are there ways to handle cases ...
0
votes
1answer
42 views

How do I classify strings with possibly no meaning?

I am quite new to text classification. Using EAST text detection model, I get multiple strings that aren't words and most often have no meaning. For example, IDs, brand names, etc. I would like to ...
0
votes
1answer
30 views

Is there any way to classify Document Image without OCR?

I have multiple invoices images which need to classify invoice types such as fright, utility, goods, etc. Is there any way to classify without OCR?
2
votes
1answer
643 views

Adding BERT embeddings in LSTM embedding Layer

I am planning to use BERT embeddings in the LSTM embedding layer instead of the usual Word2vec/Glove Embeddings. What are the possible ways to do that?
3
votes
1answer
77 views

How can I build an AI with NLP that read stories

I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet. The idea is as follows: Let's say we have a story (in the text) that has 10 ...
1
vote
1answer
40 views

Do I need to use a pre-processed dataset to classify comments?

I want to use Machine Learning for text classification, more precisely, I want to determine whether a text (or comment) is positive or negative. I can download a dataset with 120 million comments. I ...
1
vote
0answers
23 views

Conversational AI

I'm wondering if anyone knows about any projects that break down conversations for an AI. I'll explain what I mean - I've seen a lot of hardcoded responses, but nothing that allows an AI to break a ...
0
votes
0answers
65 views

How Seq2Seq with Bidirectional RNN works?

First of all the scope of the question is as follows - we have Sequence2Sequence architecture with: Decoder: Bidirectional LSTM Encoder: regular (single directional) LSTM What I know: When you ...
1
vote
1answer
51 views

How to distinguish between proper nouns and other words in NLP?

If an NLP system processes a text containing proper nouns like names, trade marks, etc. without knowing anything about the language (ie no lexicon), is it possible to recognise them?
27
votes
9answers
6k views

What is the actual quality of machine translations?

Till today I - as an AI layman - am confused by the promised and achieved improvements of automated translation. My impression is: there is still a very, very far way to go. Or are there other ...
3
votes
1answer
49 views

Will BERT embedding be always same for a given document when used as a feature extractor

When we use BERT embeddings for a classification task, would we get different embeddings everytime we pass the same text through the BERT architecture? If yes, is it the right way to use the ...
1
vote
0answers
29 views

Generate QA dataset from large text corpus

I have a corpus of a domain data in form of 10-15 books pdf and some articles and my end-goal is to make a question-answering system particular to that domain. For that, I would need a dataset on Q/A ...
3
votes
1answer
59 views

How do I identify a monologue or dialogue in a conversation?

How do I identify monologues and dialogues in a conversation (or transcript) using natural language processing? How do I distinguish between the two?
1
vote
0answers
39 views

Does the human brain use beam search for text generation?

As far as I understand, beam search is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?
0
votes
2answers
70 views

How can I implement a GAN network for text (review) generation?

How can I implement a GAN network for text (review) generation? Please, can someone guide me to resource (code) to help in text generation?
1
vote
1answer
95 views

Why can we approximate the joint probability distribution using the output vector of an LSTM?

In the paper, Contextual String Embeddings for Sequence Labeling, the authors state that \begin{equation} P(x_{0:T}) = \prod_{t=0}^T P(x_t|x_{0:t-1}) \end{equation} They also state that, in the LSTM ...
1
vote
0answers
29 views

Bert super easy implementation

I myself am not new to NLP, but for some reason I am unable to grasp purity of BERT. I have seen a ton of blogs, github repos, but none could clarify BERT usage to me. It would be helpful if you ...
1
vote
1answer
18 views

Convolutional Sequence to Sequence Learning kernel parameters

I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look ...
1
vote
1answer
25 views

What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?

Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem ...
1
vote
1answer
25 views

Having trouble figuring out how loss was calculated for SQuAD task in BERT paper

The BERT Paper https://arxiv.org/pdf/1810.04805.pdf Section 4.2 covers the SQuAD training. So from my understanding, there are two extra parameters trained, they are two vectors with the same ...
0
votes
0answers
6 views

What services are available for interpreting batches of text in order to determine their topic and/or summary?

I'm aware of the services from Microsoft and AWS, that may be able to used for such an application. Please let me know how these fare, and if other similar services exist elsewhere: https://azure....
2
votes
2answers
53 views

Do we have cross-language vector space for word embedding?

Do we have cross-language vector space for word embedding? When measure similarity for apple/Pomme/mela/Lacus/苹果/りんご, they should be the same If would be great if there's available internet service ...
1
vote
1answer
47 views

How can we create a vector space where word spelling and pronunciation can be easily compared?

In natural language processing, we can convert words to vectors (or word embeddings). In this vector space, we can measure the similarity between these word embeddings. How can we create a vector ...