Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
1answer
203 views

How to fine-tune GPT-J with small dataset

I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to fine-tune GPT-J with a small dataset of ~500 lines: ...
0
votes
1answer
26 views

What kind of NN to use to find misprints in test

I have a bunch of unique full names of users. I made pseudo-physical model to emulate misprints of desktop and mobile users (hence, fatfingering, jumpy fingers, accidentals touches of touch bar etc.) ...
1
vote
1answer
28 views

Is image machine translation done in two steps?

Suppose I have images of hand-written Japanese text. If I want to translate those images, would my ML algorithm be a 2-step model (for example, a CNN to convert the image into Japanese characters/...
0
votes
1answer
43 views

Which data representation of text as input for NLP Deep Learning models?

I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. ...
0
votes
0answers
11 views

How to use tabular data and figures when the structure of data varies?

Suppose I have several journal articles. I would like to train a binary classifier on whether the journal article is insightful. NLP models such as BERT certainly fit my need by scanning the whole ...
0
votes
2answers
82 views

Which NLP applications are based on recurrent neural networks?

Some of the NLP applications taken from this link NLP Applications: Machine Translation Speech Recognition Sentiment Analysis Question Answering Automatic Summarization Chatbots Market Intelligence ...
0
votes
1answer
60 views

Can the attention mechanism improve the performance in the case of short sequences?

I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise. ...
3
votes
1answer
134 views

How can I build an AI with NLP that read stories [closed]

I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet. The idea is as follows: Let's say we have a story (in the text) that has 10 ...
1
vote
2answers
63 views

Convolutional Sequence to Sequence Learning kernel parameters

I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look ...
1
vote
1answer
46 views

Training and Evaluating BERT and XLNET

I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice. In the project, I have been given a data set with (...
0
votes
2answers
169 views

Should I need to use BERT embeddings while tokenizing using BERT tokenizer?

I am new to BERT and NLP and I am a little confused with tokenization and word embedding. My doubt is if I use the BertTokenizer for tokenizing a sentence then do I have to compulsorily use ...
8
votes
1answer
167 views

Why are documents kept separated when training a text classifier?

Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF ...
3
votes
2answers
122 views

What is the difference between automatic transcription and automatic speech recognition?

What is the difference between automatic transcription and automatic speech recognition? Are they the same? Is my following interpretation correct? Automatic transcription: it converts the speech to ...
5
votes
1answer
226 views

Methods to tell if a question can be answered from a paragraph

I'm working on a project related to machine Q&A, using the SQuAD dataset. I've implemented a neural-net solution for finding answers in the provided context paragraph, but the system (obviously) ...
0
votes
1answer
31 views

NLP problem Phrase/Token labeling

Looking for suggestions on how to define the following NLP problem and different ways in which it can be modeled to leverage machine learning. I believe there are multiple ways to model this problem. ...
0
votes
1answer
49 views

NLP: What is expected from the output of a perfect coreference system?

For instance, consider the following piece of text: 'The father of Richard is a very nice guy. He was born in a poor family. Because of that, Richard learnt very good values. Richard is also a very ...
1
vote
1answer
163 views

Can you train Transformers sequentially?

I’m currently trying to train a BART, which is a denoising Transformer created by Facebook researchers. Here’s my Transformer code ...
1
vote
1answer
52 views

CAPTCHA based on text comprehension and random tokens

I developed a novel type of CAPTCHA based on text comprehension and random tokens. Given a task Pick the first pair of adjacent letters and a random token ...
-1
votes
0answers
15 views

How to automatically generate yes/no questions from a paragraph to train question-answer systems?

In order to train a question-answer model without the use of external labelers, it would be good to generate training questions and answers automatically from a given paragraph. For instance if I have ...
0
votes
1answer
93 views

Why would adding all the possible embeddings be "worse" than using 1D-convolutions?

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams. Why would adding all the possible embeddings, $\binom{...
-1
votes
0answers
20 views

Algorithms for classification of very short text

I am to create a classification model for texts that typically have 3 to 4 words in them. I thought of using BERT and XLNet but not sure if they are the right choice for texts that short. Are there ...
0
votes
1answer
61 views

Generating automatic sports commentary (NLG)

I am trying to develop a "simple" announcer for sports segments that mainly consists of events like goals, fouls, substitutions, and many other events that could happen in many sports. The ...
1
vote
2answers
65 views

Can I always use "encoding" and "embedding" interchangeably?

This question is restricted to the text domain only. The meaning of the word "encode" is Convert (information or instruction) into a particular form. One which performs encoding is called an ...
2
votes
3answers
155 views

Is there a relationship between Computer Algebra and NLP?

My intuition is that there is some overlap between understanding language and symbolic mathematics (e.g. algebra). The rules of algebra are somewhat like grammar, and the step-by-step arguments get ...
1
vote
1answer
77 views

Why is the sample size of stochastic gradient descent a power of 2?

I watched the video lecture of cs224: Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 2 – Word Vectors and Word Senses. They take the sample size of the window to be $2^5 = 32$ or $2^6 ...
0
votes
2answers
44 views

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of ...
11
votes
2answers
582 views

Is anybody still using Conceptual Dependency Theory?

Roger Schank did some interesting work on language processing with Conceptual Dependency (CD) in the 1970s. He then moved somewhat out of the field, being in Education these days. There were some ...
9
votes
3answers
5k views

Can AI write good jokes yet?

Just watched a recent WIRED video on virtual assistants' performance on telling jokes. They're composed by humans, but I'd like to know if AI has gotten good enough to write some.
2
votes
1answer
71 views

How to predict the "word" based on the meaning in a document?

What I mean to say is For example, if I give the meaning of Apple from the dictionary as input to the program, it should give output as Apple. Or I say My day to day job involves monitoring and ...
5
votes
2answers
1k views

How is the F1 score calculated in a question-answering system?

I have an NLP model for answer-extraction. So, basically, I have a paragraph and a question as input, and my model extracts the span of the paragraph that corresponds to the answer to the question. I ...
12
votes
1answer
1k views

Can an AI make a constructed (natural) language?

According to Wikipedia An artificial or constructed language (sometimes called a conlang) is a language that has been created by a person or small group, instead of being formed naturally as part of ...
0
votes
1answer
42 views

Building an AI that predicts the pronunciation of words

I want to create an AI that converts words to International Phonetic Alphabet (IPA), but I am not sure which architecture I am supposed to use. It is not possible to translate the characters one by ...
17
votes
2answers
337 views

What research has been done in the domain of "identifying sarcasm in text"?

Identifying sarcasm is considered one of the most difficult open-ended problems in the domain of ML and NLP/NLU. So, was there any considerable research done on that front? If yes, then what is the ...
12
votes
2answers
7k views

What are the main differences between skip-gram and continuous bag of words?

The skip-gram and continuous bag of words (CBOW) are two different types of word2vec models. What are the main differences between them? What are the pros and cons of both methods?
4
votes
1answer
301 views

What are pros and cons of Bi-LSTM as compared to LSTM?

What are the pros and cons of LSTM vs Bi-LSTM in language modelling? What was the need to introduce Bi-LSTM?
1
vote
1answer
639 views

Is there a tool to convert from the brat standoff format to CoNLL-U format? [closed]

I've been searching for a tool to convert from the brat standoff format to the CoNLL-U format, so that to use it as a parsing corpus model to the spaCy library. Can you help me?
-1
votes
1answer
706 views

How can I create my own Google duplex? [closed]

I am trying to create my own variant of Google duplex however, it won't make calls but just have a real-time conversation. My question is, where and how to start? How do I train my model with real ...
2
votes
1answer
64 views

How would you build an AI to output the primary concept of a paragraph?

My thinking is you input a paragraph, or sentence, and the program can boil it down to the primary concept(s). Example: Input: Sure, it would be nice if morality was simply a navigation toward ...
2
votes
1answer
62 views

What is the current state-of-the-art in unsupervised cross-lingual representation learning?

What is the current state-of-the-art in unsupervised cross-lingual representation learning?
3
votes
2answers
76 views

What are the advantages of Machine Learning compared to traditional programming for developing a chatbot?

I am currently building a chatbot. What I have done so far is, collected possible questions/training data/files and create a model out of it using Apache OpenNLP; the model is able to predict all the ...
5
votes
1answer
87 views

Is it possible to recognise a person based on what they have written?

I needed to make a system for recognizing people based on hundreds of texts by finding similarities in their written text grammatically or similarities between words they choose for writing. I don't ...
1
vote
2answers
82 views

What is "Word Sense Disambiguation"?

I recently came across this article which cites a paper, which apparently won the outstanding paper award in ACL 2019. The theme is that it solved a longstanding problem called Word Sense ...
2
votes
1answer
2k views

Can AI solve jumbled words?

My question is that is there any general idea on how humans solve jumbled words? I know many people will say we match it against a commonly used words checklist mentally, but it is kind of vague. Is ...
5
votes
3answers
464 views

Has an AI ever solved a detective mystery?

In detective novels, the point is often that the reader gets enough information to solve the crime themselves. This "puzzle" aspect of detective novels is part of the attraction. Often the ...
1
vote
1answer
59 views

Can Facebook's LASER be used like BERT?

Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis? From my understanding, they created an embedding that allows for similar words in different languages ...
1
vote
1answer
57 views

Which pre-processing steps are necessary for Deep Learning models to solve a document classification problem?

I have created a data set with 30.000 text documents (each text file is rather small with respect to its length), which are labelled with 0 and 1. Using this data set, I want to train machine learning ...
2
votes
1answer
139 views

How are certain machine learning models able to produce variable-length outputs given variable-length inputs?

Most machine learning models, such as multilayer perceptrons, require a fixed-length input and output, but generative (pre-trained) transformers can produce sentences or full articles of variable ...
4
votes
1answer
162 views

Can a computer identify the philosophical concept on which a given story is based?

Say you have to enter a story to a computer. Now, the computer has to identify the philosophical concept on which the story is based, say: Was it a "self-fulfilling prophecy"? Was it an ...
1
vote
0answers
30 views

Is there a way to break a piece of dialogue into components?

In many chatbots, I've seen a lot of hardcoded responses, but nothing that allows an AI to break a piece of dialogue into components (say that the speaker sounds happy or is trying to be manipulative) ...
3
votes
1answer
258 views

What are the keys and values of the attention model for the encoder and decoder in the "Attention Is All You Need" paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...

1
2 3 4 5
12