Questions tagged [natural-language-processing]

For questions related to natural language processing, which concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

How to change this RNN text classification code to become text generation code?

I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T ...
2
votes
0answers
17 views

What is the State-of-the-Art open source Voice Cloning tool right now?

I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning),...
2
votes
0answers
16 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
1answer
35 views

Why do you need to retrain GPT-2?

I'm following this tutorial, and I wonder why is there a train-step - why is it necessary? I thought the whole idea of GPT-2 is that you do not need to train it on specific text domain, as it's ...
6
votes
2answers
182 views

How would an AI learn idiomatic phrases in a natural language?

After an AI goes through the process described in How would an AI learn language?, an AI knows the grammar of a language through the process of grammar induction. They can speak the language, but they ...
-1
votes
1answer
114 views

Can a neural net learn to read?

I am a student of last year of computer engineering and lately I have been very interested in AI. Fields such as ML and DL seem very disruptive to me. A few months ago I saw an interview of Bill ...
3
votes
1answer
49 views

Techniques for sentence transformation and consistency

I'm looking for NLP techniques to transform sentences without affecting their meaning, and measures of textual consistency. For example, techniques that could transform active voice into passive voice,...
2
votes
2answers
56 views

Chatbot using Machine Learning vs Programming

I am currently building a chatbot. What I have done so far is, collected possible questions/training data/files and created a model out of it using Apache OpenNLP; the model is able to predict all the ...
2
votes
1answer
51 views

How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?

This is related to my earlier question, which I'm trying to break down into parts (this being the first). I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. ...
0
votes
0answers
19 views

Understanding how continuous bag of words method learns embedded representations

I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding: If we have a vocabulary $V$, the naive way to ...
0
votes
1answer
36 views

Preparation of input data

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different ...
2
votes
2answers
33 views

Can ELMO embeddings be used to find the n most similar sentences?

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a ...
0
votes
1answer
24 views

How to use pretrained checkpoints of BERT model on semantic text similarity task?

I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity. ...
1
vote
2answers
38 views

What is “Word Sense Disambiguation”?

I recently came across this article which cites a paper which apparently won outstanding paper in ACL 2019. The theme is that it solved a longstanding problem called Word Sense Disambiguation. What ...
2
votes
0answers
37 views

Grouped Text classification

I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph ...
1
vote
3answers
42 views

Why do language models place less importance on punctuation?

I have very outdated idea about how NLP tasks are carried out by normal RNN's, LSTM's/GRU's, word2vec, etc to basically generate some hidden form of the sentence understood by the machine. One of the ...
1
vote
0answers
6 views

How can we use Dependency Parsers for Negation detection

I am building a negation detection system. How to use dependency parsers for the same. I am using SPACY for dependency parser
0
votes
0answers
19 views

How do I do further (domain specific) pre-training with Google BERT in preparation for subsequent fine-tuning?

Another way to say this is "can you create a .ckpt file created from the final output of BERT?" Here's a recent paper that talks about additional fine-tuning.
1
vote
0answers
17 views

Language Model from missing data

I want to learn how a set of operations (my vocabulary) are composed in a dataset of algorithms (corpus). The algorithms are a sequence of higher level operations which have varying low-level ...
1
vote
0answers
47 views

Machine learning methods to identify the recipient of a document?

I need some advice on what AI methods would be suited to the identification of a recipient of a document, where the format of the documents may vary.
0
votes
0answers
20 views

How can I do entity recognition given the features?

I constructed the features from this tutorial: https://sklearn-crfsuite.readthedocs.io/en/latest/tutorial.html#features. But I did not understand the process of making the model. After doing the ...
3
votes
0answers
22 views

Can computers recognise “grouping” from voice tonality?

In human communication, tonality or tonal language play many complex information, including emotions and motives. But excluding such complex aspects, tonality serves some a very basic purpose of "...
2
votes
0answers
61 views

What is a conditional random field?

I new in machine learning, especially in Conditional Random Fields (CRF). I have read several articles and papers and in there is always associated with HMM and sequences classification. I don't ...
2
votes
0answers
26 views

Models to extract Causal Relationship between entities in a document using Natural Language Processing techniques

I am looking to extract causal relations between entities like Drug and Adverse Effect in a document. Are there any proven NLP or AI techniques to handle the same. Also are there ways to handle cases ...
0
votes
1answer
41 views

How do I classify strings with possibly no meaning?

I am quite new to text classification. Using EAST text detection model, I get multiple strings that aren't words and most often have no meaning. For example, IDs, brand names, etc. I would like to ...
0
votes
1answer
28 views

Is there any way to classify Document Image without OCR?

I have multiple invoices images which need to classify invoice types such as fright, utility, goods, etc. Is there any way to classify without OCR?
2
votes
1answer
445 views

Adding BERT embeddings in LSTM embedding Layer

I am planning to use BERT embeddings in the LSTM embedding layer instead of the usual Word2vec/Glove Embeddings. What are the possible ways to do that?
3
votes
1answer
76 views

How can I build an AI with NLP that read stories

I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet. The idea is as follows: Let's say we have a story (in the text) that has 10 ...
1
vote
1answer
40 views

Do I need to use a pre-processed dataset to classify comments?

I want to use Machine Learning for text classification, more precisely, I want to determine whether a text (or comment) is positive or negative. I can download a dataset with 120 million comments. I ...
1
vote
0answers
23 views

Conversational AI

I'm wondering if anyone knows about any projects that break down conversations for an AI. I'll explain what I mean - I've seen a lot of hardcoded responses, but nothing that allows an AI to break a ...
0
votes
0answers
51 views

How Seq2Seq with Bidirectional RNN works?

First of all the scope of the question is as follows - we have Sequence2Sequence architecture with: Decoder: Bidirectional LSTM Encoder: regular (single directional) LSTM What I know: When you ...
1
vote
1answer
45 views

How to distinguish between proper nouns and other words in NLP?

If an NLP system processes a text containing proper nouns like names, trade marks, etc. without knowing anything about the language (ie no lexicon), is it possible to recognise them?
27
votes
9answers
6k views

What is the actual quality of machine translations?

Till today I - as an AI layman - am confused by the promised and achieved improvements of automated translation. My impression is: there is still a very, very far way to go. Or are there other ...
3
votes
1answer
45 views

Will BERT embedding be always same for a given document when used as a feature extractor

When we use BERT embeddings for a classification task, would we get different embeddings everytime we pass the same text through the BERT architecture? If yes, is it the right way to use the ...
1
vote
0answers
26 views

Generate QA dataset from large text corpus

I have a corpus of a domain data in form of 10-15 books pdf and some articles and my end-goal is to make a question-answering system particular to that domain. For that, I would need a dataset on Q/A ...
3
votes
1answer
57 views

How do I identify a monologue or dialogue in a conversation?

How do I identify monologues and dialogues in a conversation (or transcript) using natural language processing? How do I distinguish between the two?
1
vote
0answers
39 views

Does the human brain use beam search for text generation?

As far as I understand, beam search is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?
0
votes
2answers
70 views

How can I implement a GAN network for text (review) generation?

How can I implement a GAN network for text (review) generation? Please, can someone guide me to resource (code) to help in text generation?
1
vote
1answer
92 views

Why can we approximate the joint probability distribution using the output vector of an LSTM?

In the paper, Contextual String Embeddings for Sequence Labeling, the authors state that \begin{equation} P(x_{0:T}) = \prod_{t=0}^T P(x_t|x_{0:t-1}) \end{equation} They also state that, in the LSTM ...
1
vote
0answers
28 views

Bert super easy implementation

I myself am not new to NLP, but for some reason I am unable to grasp purity of BERT. I have seen a ton of blogs, github repos, but none could clarify BERT usage to me. It would be helpful if you ...
1
vote
1answer
14 views

Convolutional Sequence to Sequence Learning kernel parameters

I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look ...
1
vote
1answer
25 views

What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?

Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem ...
1
vote
1answer
25 views

Having trouble figuring out how loss was calculated for SQuAD task in BERT paper

The BERT Paper https://arxiv.org/pdf/1810.04805.pdf Section 4.2 covers the SQuAD training. So from my understanding, there are two extra parameters trained, they are two vectors with the same ...
0
votes
0answers
6 views

What services are available for interpreting batches of text in order to determine their topic and/or summary?

I'm aware of the services from Microsoft and AWS, that may be able to used for such an application. Please let me know how these fare, and if other similar services exist elsewhere: https://azure....
2
votes
2answers
53 views

Do we have cross-language vector space for word embedding?

Do we have cross-language vector space for word embedding? When measure similarity for apple/Pomme/mela/Lacus/苹果/りんご, they should be the same If would be great if there's available internet service ...
1
vote
1answer
47 views

How can we create a vector space where word spelling and pronunciation can be easily compared?

In natural language processing, we can convert words to vectors (or word embeddings). In this vector space, we can measure the similarity between these word embeddings. How can we create a vector ...
0
votes
0answers
26 views

How do I create syntactically correct sentences given several words?

Is there an AI application that can produce syntactically (and semantically) correct sentences given a bag of words? For example, suppose I am given the words "cat", "fish", and "lake", then one ...
1
vote
1answer
35 views

Can AI help summarize article or abstract sentence keyword?

I'm wondering if AI now can help us abstract summary or general idea of long article, for example novel or historical stories, or abstract most important keyword from sentence; Would you please tell ...
2
votes
0answers
50 views

ChatBot applications [Academia]

I am in education and I'm wondering about the use of chatbot-like tools to facilitate automated discussions among students. The chatbot domain of knowledge would be purposely restricted to a specific ...
1
vote
0answers
64 views

How does bidirectional encoding allow the predicted word to indirectly “see itself”?

Before the release of BERT we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that’s ...