Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

How to find the accuracy of LDA model?

I am working on topic modeling using the latent Dirichlet allocation model. I have a dataset that contains tweets and topics corresponding to these tweets. In total, there are 65 different topics. I ...
2
votes
0answers
25 views

Why do Transformers have a sequence limit at inference time?

As far as I understand, Transformer's time complexity increases quadratically with respect to the sequence length. As a result, during training to make training feasible, a maximum sequence limit is ...
0
votes
0answers
20 views

Using NLP embeddings in trajectory or motion planning of autonomous system

I am planning to use textual rules like traffic rules in motion planning of autonomous car. I can think of using BERT like models to generate embeddings and then use these embeddings for motion or ...
0
votes
0answers
28 views

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of ...
0
votes
1answer
20 views

What does it mean to apply decomposition at inference-time in a machine translation system?

I'm reading this paper for sub-character decomposition for logographic languages and the authors mention decomposition at inference-time. They're using Transformer architecture. More specifically, the ...
2
votes
1answer
34 views

General approaches in text encoding and labelling for NLP [closed]

What are the approaches of encoding text data? I would be glad to hear some summarization from experienced persons. And are there any solutions accepting words outside the vocabulary and including ...
0
votes
0answers
24 views

How to fine-tune GPT-J with small dataset

I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to fine-tune GPT-J with a small dataset of ~500 lines: ...
0
votes
0answers
19 views

Which AI algorithm to use for identifying API for a specific use from a list of APIs?

We have a legacy code solution in C#. We have to change the code so that it fetches internal data via APIs and not via DB calls. E.g. if the current code GETS Payment object from DB, we have to ...
0
votes
0answers
13 views

Getting consistent answers from Blenderbot

recently, there was a good discussion online about chatbots like the Blenderbot from Facebook. I am new to this area and i understand that blenderbot uses generative models. I believe it is possible ...
0
votes
1answer
23 views

Is PositionalEncoding needed for using Transformer models correctly?

I am trying to make a model that uses a Transformer to see the relationship between several data vectors, but the order of the data is not relevant in this case, so I am not using the Positional ...
3
votes
1answer
68 views

Why do we multipy context_size with embedding_dim? (PyTorch)

I've been using Tensorflow and just started learning PyTorch. I was following the tutorial: https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-...
0
votes
0answers
18 views

Find words in a raw text that are similar to a predefined array of words

I need a beginner advice regarding a simple application where I have a very limited set of topics in which i need to identify inside a user's natural language text/sentence. For example, one of the ...
1
vote
1answer
42 views

What is the Intermediate (dense) layer in between attention-output and encoder-output dense layers within transformer block in PyTorch implementation?

In PyTorch, transformer (BERT) models have an intermediate dense layer in between attention and output layers whereas the BERT and Transformer papers just mention the attention connected directly to ...
0
votes
0answers
13 views

How might I extend named entity recognition to recognise sentence or phrases as belonging to certain classes, such as commentary, opinion etc?

Suppose I have a news report such as THE ARCE BATTALION COMMAND HAS REPORTED THAT ABOUT 50 PEASANTS OF VARIOUS AGES HAVE BEEN KIDNAPPED BY TERRORISTS OF THE FARABUNDO MARTI NATIONAL LIBERATION FRONT [...
0
votes
1answer
26 views

Given the immaturity of NLP tools for non-English languages, should I first translate the non-English language to English before text pre-processing?

For non-English languages (in my case Portuguese), what is the best approach? Should I use the not-so-complete tools in my language, or should I translate the text to English, and after using the ...
1
vote
0answers
30 views

Why does research on faster Transformers focus on the query-key product?

A lot of recent research on Transformers has been devoted to reducing the cost of the self-attention mechanism: $$\text{softmax}\left(\frac{Q K^T}{\sqrt{d}} \right)V,$$ As I understand it, the runtime,...
1
vote
1answer
41 views

Probability that two words appear in the same sentence

How can I know if two words are likely to appear in the same sentence in (British) English (or English in general to enhance the chance of getting a result). As I don't have access to a powerful ...
1
vote
1answer
24 views

A recommender system based on millions of fields including text and number

I want to train a model based on millions of fields, including text and number, that are stored in a SQL database and recommend a perfect match based on some inputs. Now, which algorithm is the best ...
0
votes
1answer
19 views

How to reduce the number of clusters produced by the Markov Clustering Algorithm?

I have used the Markov Clustering Algorithm (MCL) to cluster tweets, based on their similarity. However, I got a too high number of clusters, and most of the clusters have only one tweet. Any ...
0
votes
0answers
12 views

Identify Merchants from Transaction Dataset

I have a transaction dataset, each transaction is in an unstructured format. The objective is to identify merchant from each transaction. If we look it from NER point of view, there would be problem ...
2
votes
1answer
70 views

How to generate a response while considering past questions as well?

User: What is the tallest mountain? Agent: Everest User: Where is it located? # Agent hears: "Where is Everest located?" Agent: Nepal I want to be able ...
1
vote
0answers
26 views

Is Speech to Speech with changing the voice to a given other voice possible?

Background: I am working on a research project to use (demonstrate) the possibilities of Machine Learning and AI in artistic projects. One thing we are exploring is demonstrating deep fakes on stage. ...
1
vote
0answers
22 views

NLP problem Phrase/Token labeling

Looking for suggestions on how to define the following NLP problem and different ways in which it can be modeled to leverage machine learning. I believe there are multiple ways to model this problem. ...
1
vote
2answers
37 views

Why do the authors of the T5 paper say that the "architectural changes are orthogonal to the experimental factors"?

Here's a quote from the T5 paper (T5 stands for "Text-to-Text Transfer Transformer") titled Exploring the Limits of Transfer Learning with a Unified Text-...
0
votes
1answer
32 views

Identify whether two companies are the same

I am trying to solve a problem where I need to map multiple variations of a company name to a single name. For example: say I have a company named ...
0
votes
0answers
25 views

Can I always use "encoding" and "embedding" interchangeably?

This question is restricted to text domain only. The meaning of word "encode" is Convert (information or an instruction) into a particular form. One which performs encoding is called encoder....
1
vote
0answers
28 views

Is there something like person-specific sentiment analysis?

Sentiment analysis, as we know, measures "Cake sucks" as say -0.4, and "Cake is great" as 0.7. What I'm looking for is something a bit different like so: Given input text data ...
0
votes
0answers
11 views

Is Byte-Pair tokenization sufficient to handle arbitrary numbers?

I'm asking because I need to use NLP in a context where 'tokens' even exist (e.g. primarily numbers & some random text), and I was wondering if in practice byte-pair tokenization is sufficient to ...
1
vote
2answers
37 views

What is meant by a "relevant document" in NLP?

In natural language processing, I came across the concept of "relevant document" several times. And several analytical formulas, such as precision, recall are based on the relevant documents....
1
vote
0answers
23 views

Alternative chatbot service with export intent/entities function like DialogFlow? [closed]

I've been searching for a chatbot-building website with exportable intents/entities to a particular format (Spreadsheet, CSV, JSON, etc.). But the chatbots I have found so far like Flow.ai or ChatFuel ...
0
votes
1answer
19 views

Is there any difference between the phrases "text representation" and "text feature representation"?

Text representation, in simple words, is representing text in sensible numeric form. You can read in detail from the following paragraph Text representation is one of the fundamental problems in text ...
0
votes
1answer
59 views

Get the name of a merchant from records

I have a bunch of bank transaction records from which I want to extract merchants' names. In a few subsets of these records, the structure of the string is the same within the subset with only the ...
1
vote
1answer
22 views

Bag of Tricks: n-grams as additional features?

I've been playing with PyTorch's nn.EmbeddingBag for sentence classification for about a month. I've been doing some feature engineering, playing with different ...
0
votes
0answers
18 views

Is there a survey that describes the most effective approaches for an answer retrieval problem?

I have a dataset that contains pairs of a question and an answer. My problem is to train a model that can search for the right answer from the pool of my answers given the newly input question, so ...
0
votes
2answers
25 views

What is the reason for a training loss that drops but validation that NEVER does

I've been working on learning about NLP via a beginners competition on Kaggle. I first trained a model with an embedding layer and then a simple linear layer. I actually got way better than a flip of ...
1
vote
1answer
35 views

Training and Evaluating BERT and XLNET

I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice. In the project, I have been given a data set with (...
0
votes
1answer
27 views

An online editor that allows data labeling format [closed]

I have a set of students (~20) that will work on annotating data for an NLP project. The annotation task will be as in the following: ...
0
votes
0answers
12 views

Performance metric for multi aspect extraction and sentiment analysis

I created a model that extracts aspects from reviews and predicts the associated sentiments. I'm now trying to evaluate the model. I tried many different approaches because there are no real true ...
0
votes
0answers
18 views

How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?

I read the tutorial of LSTM from here. However, I have certain doubts that I need to address. Since we use true labels and do not remove anything from the original data, then how is it possible for ...
1
vote
1answer
340 views

Isn't attention mask for BERT model useless?

I have just dived into deep learning for NLP, and now I'm learning how the BERT model works. What I found odd is why the BERT model needs to have an attention mask. As clearly shown in this tutorial ...
0
votes
1answer
36 views

Building an AI that predicts the pronunciation of words

I want to create an AI that converts words to International Phonetic Alphabet (IPA), but I am not sure which architecture I am supposed to use. It is not possible to translate the characters one by ...
0
votes
0answers
8 views

What next after LDA and NMF for topic modelling (with Python)?

I have used NMF and LDA for topic modelling in Python, with what I would call good results with NMF, and poor results with LDA. My data is highly domain-specific, with a lot of unique/specific ...
0
votes
1answer
44 views

The model learns well, but the validation decreases over time [closed]

I have trained a model for four days. I noticed a behaviour quite strange/unnatural. During the training, the score and loss look like this: However, when I see the validation score, I got: It seems ...
0
votes
0answers
3 views

Ensure trained word embeddings get high similarity with particular words

I am trying out my hand at training a Word2Vec model using gensim. I made a simple training file that basically had just one line repeated multiple times ...
0
votes
0answers
7 views

Part of Speech and word embedding on a mobile phone1

I have a model on Spacy that does part of speech recognition and then makes a smart average of words for each part of speech: subject, verb, object... I would like to deploy it in a mobile app that ...
0
votes
1answer
13 views

Language Processing: Determine if one paragraph is relevant to another paragraph

Context: I want to determine if someone's written review contains content that is relevant to a paragraph that they are reviewing. To do so, I am trying to determine if one paragraph is relevant to ...
0
votes
1answer
16 views

Semantic-based evaluation of translations instead of BLEU

I have a text generation model and I want to evaluate its output by comparing it to a set of gold human-annotated references. I went through machine-translation metrics and I found that BLEU is used ...
2
votes
1answer
34 views

How to measure the similarity the pronunciation of two words?

I would like to know how I could measure the pronunciation of two words. These two words are quite similar and differ only in one vowel. I know there is, e.g., the Hamming distance or the Levenshtein ...
0
votes
0answers
20 views

How to Study Improving In-depth Reading Comprehension?

There are multiple datasets for machine comprehension tasks such as SQuAD. However, most of the questions are straightforward. One can find the answers easily by using the find feature of the browser ...
1
vote
1answer
44 views

How would the probability of a document $P(d)$ be computed in the Naive Bayes classifier?

In naive Bayes classification, we estimate the class of a document as follows $$\hat{c} = \arg \max_{c \in C} P(c \mid d) = \arg \max_{c \in C} \dfrac{ P(d \mid c)P(c) }{P(d)} $$ It has been said in ...

1
2 3 4 5
12