Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

228 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

XLMRoberta loss remains constant over iterations for TokenClassification task

I have created a simple XLMRoberta model for token classification. The task is to predict the quality of translation for each token/word. The data looks something like this, where the first sentence ...
0
votes
0answers
26 views

What are the types of inputs used for RNN in literature given sentences?

Suppose there are $m$ sentences in a text file and the number of distinct words is equal to $n$. The goal is to get word embeddings using RNN. We know that it is impossible to pass any word, which is ...
0
votes
0answers
16 views

Are there any inference memory requirement tables for Hugging Face transformers?

Hugging Face has a very large list of supported transformers. They provide a table which gives the status on whether or not a transformer has a slow tokenizer, a fast tokenizer, PyTorch support, ...
0
votes
0answers
20 views

Fine tuning BERT for token level classification

I want to try self-supervised and semi-supervised learning for my task, which relates to token-wise classification for the 2 sequences of sentences (source and translated text). The labels would be ...
0
votes
1answer
26 views

What kind of NN to use to find misprints in test

I have a bunch of unique full names of users. I made pseudo-physical model to emulate misprints of desktop and mobile users (hence, fatfingering, jumpy fingers, accidentals touches of touch bar etc.) ...
0
votes
0answers
13 views

Is there a way to train Doc2Vec on a corpus of docs and be able to take a novel doc and see how similar it is to the training corpus?

I have a project idea, where I train a bunch of documents on Doc2Vec, and then take a novel, input doc, and ideally be able to be told how similar it is to the docs supplied for training as a whole or ...
0
votes
0answers
13 views

How to find the accuracy of LDA model?

I am working on topic modeling using the latent Dirichlet allocation model. I have a dataset that contains tweets and topics corresponding to these tweets. In total, there are 65 different topics. I ...
0
votes
2answers
44 views

How much labelling is required for NER with SpaCy?

I have transaction data and I would like to extract the merchant from the transaction description. I am new to this but I just came across Named Entity Recognition and SpaCy. I have hundreds of ...
0
votes
1answer
212 views

How to fine-tune GPT-J with small dataset

I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to fine-tune GPT-J with a small dataset of ~500 lines: ...
0
votes
0answers
19 views

Which AI algorithm to use for identifying API for a specific use from a list of APIs?

We have a legacy code solution in C#. We have to change the code so that it fetches internal data via APIs and not via DB calls. E.g. if the current code GETS Payment object from DB, we have to ...
0
votes
0answers
18 views

Getting consistent answers from Blenderbot

recently, there was a good discussion online about chatbots like the Blenderbot from Facebook. I am new to this area and i understand that blenderbot uses generative models. I believe it is possible ...
0
votes
0answers
22 views

Find words in a raw text that are similar to a predefined array of words

I need a beginner advice regarding a simple application where I have a very limited set of topics in which i need to identify inside a user's natural language text/sentence. For example, one of the ...
0
votes
0answers
14 views

How might I extend named entity recognition to recognise sentence or phrases as belonging to certain classes, such as commentary, opinion etc?

Suppose I have a news report such as THE ARCE BATTALION COMMAND HAS REPORTED THAT ABOUT 50 PEASANTS OF VARIOUS AGES HAVE BEEN KIDNAPPED BY TERRORISTS OF THE FARABUNDO MARTI NATIONAL LIBERATION FRONT [...
0
votes
1answer
23 views

How to reduce the number of clusters produced by the Markov Clustering Algorithm?

I have used the Markov Clustering Algorithm (MCL) to cluster tweets, based on their similarity. However, I got a too high number of clusters, and most of the clusters have only one tweet. Any ...
0
votes
0answers
13 views

Identify Merchants from Transaction Dataset

I have a transaction dataset, each transaction is in an unstructured format. The objective is to identify merchant from each transaction. If we look it from NER point of view, there would be problem ...
0
votes
1answer
32 views

NLP problem Phrase/Token labeling

Looking for suggestions on how to define the following NLP problem and different ways in which it can be modeled to leverage machine learning. I believe there are multiple ways to model this problem. ...
0
votes
0answers
11 views

Is Byte-Pair tokenization sufficient to handle arbitrary numbers?

I'm asking because I need to use NLP in a context where 'tokens' even exist (e.g. primarily numbers & some random text), and I was wondering if in practice byte-pair tokenization is sufficient to ...
0
votes
0answers
18 views

Is there a survey that describes the most effective approaches for an answer retrieval problem?

I have a dataset that contains pairs of a question and an answer. My problem is to train a model that can search for the right answer from the pool of my answers given the newly input question, so ...
0
votes
0answers
12 views

Performance metric for multi aspect extraction and sentiment analysis

I created a model that extracts aspects from reviews and predicts the associated sentiments. I'm now trying to evaluate the model. I tried many different approaches because there are no real true ...
0
votes
0answers
18 views

How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?

I read the tutorial of LSTM from here. However, I have certain doubts that I need to address. Since we use true labels and do not remove anything from the original data, then how is it possible for ...
0
votes
1answer
42 views

Building an AI that predicts the pronunciation of words

I want to create an AI that converts words to International Phonetic Alphabet (IPA), but I am not sure which architecture I am supposed to use. It is not possible to translate the characters one by ...
0
votes
0answers
10 views

What next after LDA and NMF for topic modelling (with Python)?

I have used NMF and LDA for topic modelling in Python, with what I would call good results with NMF, and poor results with LDA. My data is highly domain-specific, with a lot of unique/specific ...
0
votes
0answers
3 views

Ensure trained word embeddings get high similarity with particular words

I am trying out my hand at training a Word2Vec model using gensim. I made a simple training file that basically had just one line repeated multiple times ...
0
votes
1answer
16 views

Semantic-based evaluation of translations instead of BLEU

I have a text generation model and I want to evaluate its output by comparing it to a set of gold human-annotated references. I went through machine-translation metrics and I found that BLEU is used ...
0
votes
0answers
20 views

How to Study Improving In-depth Reading Comprehension?

There are multiple datasets for machine comprehension tasks such as SQuAD. However, most of the questions are straightforward. One can find the answers easily by using the find feature of the browser ...
0
votes
0answers
15 views

Is it possible to identify multiple queries/intents in an email, check if the reply has addressed all of those queries before sending email?

An email may contain multiple questions related to similar or distinct topics. The person responding the email needs assistance in detecting and informing if all of the questions have been addressed ...
0
votes
0answers
11 views

How does the bigram terms are contributing to sophisticated version of linear interpolation?

While studying about linear interpolation technique in natural language processing to deal with less frequent $n-$gram. I came across a sophisticated version of linear interpolation. The simple and ...
0
votes
0answers
30 views

What exactly "adjusted count" tells us in Laplace smoothing?

The probability/frequency of a word $w_i$ in an $N$-word corpus is given by $$p(w_i) = \dfrac{c_i}{N}$$ where $c_i$ is the number of times the word $w_i$ appears in the corpus. Suppose there are $V$ ...
0
votes
0answers
16 views

Is there any dataset to convert text to sign language?

I'm going to start working on one university project and I would like to ask a question regarding it. My project is about "Sign language synthesis from NLP" and I need to develop an ...
0
votes
1answer
25 views

Extracting keywords from messages

I'm starting a project where I want to extract keywords from given messages. The keywords are for example something like: "hard disk", "watch" or other technical components. I'm ...
0
votes
0answers
15 views

Does the transformers model (in "Attention is All You Need") exclude the encoder in language modelling tasks?

The language model I am referring to is the one outlined in "Attention is All You Need": My understanding is that when the task is translation, the encoder's input could be "Hi, my ...
0
votes
0answers
15 views

MBART and better Domain Specific Translations Using Masks?

I'm implementing https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt for translations as it has shown promising results but I wanted to see if there was a way to translate specific parts ...
0
votes
0answers
18 views

Understanding manual test cases

I have a large number of manual test cases which are designed to test a web application's functionality. I want to automate these and I have been exploring NLP to structurize the test cases. These ...
0
votes
0answers
19 views

Open Domain Chatbot with intervention

As per my knowledge, there are 2 type of Chatbots- Open Domain and Closed Doman Intent Based. I was wondering how are conversational agents created such that can converse as if they are Open Domain, ...
0
votes
0answers
9 views

How can I design a machine translation model that produces a mapping between the words in the source and target sentences?

I have a dataset of sentences of language X and Y X Y abc def lang xyz pqrt mno uages I want to have an output as a table with word-by-word translation (...
0
votes
0answers
13 views

ML model to predict values from text (a lot of training training data)

I have around 1M entries of the type: id | big5_openness | big5_conscientiousness | big5_extraversion | big5_agreeableness | big5_neuroticism | input_text Where <...
0
votes
0answers
13 views

How to Approach a Conversation Detector?

I am currently looking for a way to detect if a conversation is occurring. The meaning of the conversation is not important for this case. One approach that seems viable is to try to detect a voice ...
0
votes
0answers
15 views

Does it make sense to generate sentences with Transformer's encoder?

Quite a few vision+language papers pretrain BERT-based model with image-text data and finetune for image captioning task. But there is no decoder involved to generate sentences. Does that make sense? ...
0
votes
0answers
31 views

Do the training and test datasets need to be equally preprocessed as one whole dataset?

I have developed, trained and tested an NLP model. It is persisted in a pickle file. The model contains the data preprocessing function that includes text cleaning and new features engineered with ...
0
votes
0answers
27 views

Speech diarization for a conversation detector: A good idea or not?

I am trying to write a program in which an ai can detect whether a conversation is occurring or not. The ai does not need to transcribe words or have any meaning about the conversation, simply if one ...
0
votes
0answers
17 views

Can I use the phi coefficient to compare predictions by two different classifiers?

Can I use the Matthews correlation coefficient (aka phi coefficient) to compare predictions by two different classifiers? That is, is this code correct if I want to check how diverse/correlated my two ...
0
votes
0answers
124 views

Why does KNN algorithm perform better on Word2Vec than on TF-IDF vector representation?

I am doing a project on multi-class text classification and could do with some advice. I have a dataset of reviews that are classified into 7 product categories. Firstly, I create a term-document ...
0
votes
0answers
20 views

How can I prune BERT layers

I would like to finetune BERT on SQuAD and then evaluate the output from each layer (so from using 1 layer to using all 12). I know you can prune heads using Huggingface but was wondering how could ...
0
votes
0answers
21 views

Word2vec - CBOW and Skipgram comparative study for big data

Word2vec - CBOW and Skipgram comparative study As a part of my thesis I am creating a comparative study of CBOW and Skipgram for big data. My input is a Wikipedia dump and I have created a single .txt ...
0
votes
0answers
9 views

What is the recommended way to retokenize ontonotes to wordpiece?

I am using ontonotes5 dataset which is already tokenize. But since I want to use Bert. I want to tokenize it myself, keeping all the annotations correct.
0
votes
0answers
34 views

How do the trainable projection layer used in PRADO and pQRNN work?

Trainable projection layers are said to be a very powerful thing but after reading: https://www.aclweb.org/anthology/D19-1506.pdf https://arxiv.org/pdf/2101.08890.pdf I don't understand how it works....
0
votes
0answers
31 views

Why (not) using pre-processing before using Transformer models?

Regarding the use of pre-processing techniques before using Transformers models, I read this post that apparently says that these measures are not so necessary nor interfere so much in the final ...
0
votes
0answers
20 views

Text analysis or just statistical model?

For a college class, I have to train an ML model. I have an idea for a project but I am unsure if it is wise to attempt to solve this problem with ML as opposed to maybe more conventional methods of ...
0
votes
0answers
17 views

Extracting keywords from documents and predicting future keywords

I apologise for the newbie question! I've played with ML before, but at a very superficial level. I now have an interesting problem to solve and I wondered whether you could share some thoughts - I ...
0
votes
0answers
36 views

Adding BERT embeddings in BiLSTM embedding layer

I am want to use BERT embeddings in the BiLSTM embedding layer instead of Word2Vec or FastText Embeddings. There is any code to do that?