People who code: we want your input. Take the Survey

Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Why (commonly/only) $\log$ to squash frequencies?

Term frequency and inverse document frequency are well known terms in information retrieval. I am presenting the definitions for both from p:12,13 of Vector Semantics and Embeddings On term frequency ...
0
votes
0answers
6 views

Does the transformers model (in “Attention is All You Need”) exclude the encoder in language modelling tasks?

The language model I am referring to is the one outlined in "Attention is All You Need": My understanding is (please correct me if I am wrong) that when the task is translation, the encoder'...
0
votes
0answers
28 views

Building a knowledge graph for programming languages

I've been interested in semantic similarity and 'translation' between programming languages. Not only in syntax for basics like for loops, declaring classes, etc. but also for more abstract concepts ...
0
votes
0answers
9 views

MBART and better Domain Specific Translations Using Masks?

I'm implementing https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt for translations as it has shown promising results but I wanted to see if there was a way to translate specific parts ...
0
votes
0answers
18 views

Understanding manual test cases

I have a large number of manual test cases which are designed to test a web application's functionality. I want to automate these and I have been exploring NLP to structurize the test cases. These ...
0
votes
1answer
19 views

Changing a CNN-LSTM image captioning architecture to use BiLSTMs

Currently I'm dealing with an assignment that made us implement the network mentioned in this paper. The network has an architecture similar to this: As you can see it uses a Unidirectional RNN (in ...
2
votes
1answer
36 views

Which AI techniques are there that combine multiple models to make sense of data at different stages?

I have been working to design a system that uses multiple machine learning models to make sense of data that is dynamically webscraped. Each AI would handle a specific task, for example: An AI model ...
1
vote
0answers
17 views

Why are BERT embeddings interpreted as representations of the corresponding words?

It's often assumed in literature that BERT embeddings are contextual representations of the corresponding word. That is, if the 5th word is "cold", then the 5th BERT embedding is a ...
0
votes
0answers
13 views

Open Domain Chatbot with intervention

As per my knowledge, there are 2 type of Chatbots- Open Domain and Closed Doman Intent Based. I was wondering how are conversational agents created such that can converse as if they are Open Domain, ...
1
vote
0answers
30 views

How do sparse word embeddings fail to capture synonymy?

While reading some explanations of why dense word embeddings work better than sparse word embeddings, the following statement has been given in the chapter Vector Semantics and Embeddings, showing a ...
1
vote
1answer
37 views

What is an “input embedding” in the context of NLP?

When reading about NLP, I saw it said that "input embeddings" are a main element of encoder-decoder learning frameworks for sequence modelling. What is an "input embedding" in the ...
1
vote
0answers
14 views

Can unsupervised models learn something from cat vocalizations?

I love cats, and over the years have noticed that they have recurrent patterns of vocalizations. For example, upon seeing a bird, a cat may start chittering, but the same cat would never chitter at ...
0
votes
1answer
23 views

How to perform multi-class text classification with a dataset of 80 documents?

I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags. How can I perform multi-class text classification with such a small ...
0
votes
0answers
9 views

How can I design a machine translation model that produces a mapping between the words in the source and target sentences?

I have a dataset of sentences of language X and Y X Y abc def lang xyz pqrt mno uages I want to have an output as a table with word-by-word translation (...
0
votes
1answer
46 views

Seeking advice: Building a NLP in C++ [closed]

Context: I'm taking an advanced C++ course and there is a large 2 month project of my choosing that I'll be starting soon. I want to build a NLP that can do text completion, but I'm new to this area ...
0
votes
1answer
22 views

How to build a custom morphological analyser for translation system

I want to build a machine translation system from English to Georgian. Georgian is a language similar (and simpler) to the Russian language. its syntax looks like base + suffix, only suffix changes, ...
0
votes
2answers
63 views

Book(s) for text embedding

Text here refers to either character or word or sentence. Is there any recent textbook that encompasses from classical methods to the modern techniques for embedding texts? If a single textbook is ...
1
vote
0answers
13 views

Current extensions of the “Turing Test”?

In 2014 it was widely reported that the Turing Test had been passed, and that this was a major AI milestone. See: Computer AI passes Turing test in 'world first [BBC]; Turing Test Success Marks ...
0
votes
0answers
12 views

ML model to predict values from text (a lot of training training data)

I have around 1M entries of the type: id | big5_openness | big5_conscientiousness | big5_extraversion | big5_agreeableness | big5_neuroticism | input_text Where <...
1
vote
0answers
10 views

How to detect the description of spine segments in short text using a neural network?

The input data is a set of text chunks containing the description of the pathology or the surgical procedure: For instance: Tere is a lumbar stenosis L3/4 Patient ist suffering from [...], MRI and X ...
0
votes
0answers
12 views

How to Approach a Conversation Detector?

I am currently looking for a way to detect if a conversation is occurring. The meaning of the conversation is not important for this case. One approach that seems viable is to try to detect a voice ...
0
votes
0answers
10 views

Does it make sense to generate sentences with Transformer's encoder?

Quite a few vision+language papers pretrain BERT-based model with image-text data and finetune for image captioning task. But there is no decoder involved to generate sentences. Does that make sense? ...
0
votes
0answers
26 views

Do the training and test datasets need to be equally preprocessed as one whole dataset?

I have developed, trained and tested an NLP model. It is persisted in a pickle file. The model contains the data preprocessing function that includes text cleaning and new features engineered with ...
6
votes
1answer
2k views

What language is the GPT-3 engine written in?

I know that the API is python based, but what's the gpt-3 engine written in mostly? C? C++? I'm having some trouble finding this info.
0
votes
0answers
19 views

Speech diarization for a conversation detector: A good idea or not?

I am trying to write a program in which an ai can detect whether a conversation is occurring or not. The ai does not need to transcribe words or have any meaning about the conversation, simply if one ...
0
votes
0answers
15 views

Can I use the phi coefficient to compare predictions by two different classifiers?

Can I use the Matthews correlation coefficient (aka phi coefficient) to compare predictions by two different classifiers? That is, is this code correct if I want to check how diverse/correlated my two ...
0
votes
0answers
23 views

Why does KNN algorithm perform better on Word2Vec than on TF-IDF vector representation?

I am doing a project on multi-class text classification and could do with some advice. I have a dataset of reviews that are classified into 7 product categories. Firstly, I create a term-document ...
1
vote
0answers
10 views

Training seq2seq translation model with one source and multiple target

So basically I'm training a sequence to sequence model that translates English sentences to Arabic sentences. I'm using the data provided by Anki @ manythings. I realized that some of the sentences in ...
0
votes
0answers
11 views

How can I prune BERT layers

I would like to finetune BERT on SQuAD and then evaluate the output from each layer (so from using 1 layer to using all 12). I know you can prune heads using Huggingface but was wondering how could ...
0
votes
1answer
23 views

Creating a NLP driven chatbot [closed]

I would like to create a chat bot for an e-commerce website that sells a wide range of general merchandize items, from t-shirts, jumpers to calculators. Its primary objective is to develop a Q&A ...
0
votes
2answers
123 views

Is there any specific SW framework, libraries or algorithms (supported by any theory) designed for implementing a practical AGI system? [closed]

Any (AGI)-KERAS like libraries? Any deep-learning framework to develop AGI applications? Existing frameworks/algorithms used in NN, NLP, ML, etc are not enough in my opinion. In my opinion any ...
0
votes
0answers
17 views

Word2vec - CBOW and Skipgram comparative study for big data

Word2vec - CBOW and Skipgram comparative study As a part of my thesis I am creating a comparative study of CBOW and Skipgram for big data. My input is a Wikipedia dump and I have created a single .txt ...
2
votes
0answers
13 views

Embedding from Transformer-based model from paragraph or documnet (like Doc2Vec)

I have a set of data that contains the different lengths of sequences. On average the sequence length is 600. The dataset is like this: ...
0
votes
0answers
8 views

What is the recommended way to retokenize ontonotes to wordpiece?

I am using ontonotes5 dataset which is already tokenize. But since I want to use Bert. I want to tokenize it myself, keeping all the annotations correct.
1
vote
0answers
23 views

What is the meaning of “Our current objective weights every token equally and lacks a notion of what is most important to predict” in the GPT-3 paper?

On page 34 of OpenAI's GPT-3, there is a sentence demonstrating the limitation of objective function: Our current objective weights every token equally and lacks a notion of what is most important to ...
2
votes
0answers
12 views

How to Select Model Parameters for Transformer (Heads, number of layers, etc)

Is there a general guideline on how the Transformer model parameters should be selected, or the range of these parameters that should be included in a hyperparameter sweep? Number of heads Number of ...
2
votes
0answers
55 views

Extracting “hidden” costs from financial statements using NLP

I'm designing a NLP model to extract various kinds of "hidden" expenses from 10-K and 10-Q financial statements. I've come up with about 7 different expense categories (restructuring costs, ...
0
votes
0answers
20 views

How do the trainable projection layer used in PRADO and pQRNN work?

Trainable projection layers are said to be a very powerful thing but after reading: https://www.aclweb.org/anthology/D19-1506.pdf https://arxiv.org/pdf/2101.08890.pdf I don't understand how it works....
-1
votes
1answer
24 views

Extracting values from text based on keywords

I am trying to read a PDF file and put it in Python string and trying to fetch information based on keywords. The text here is completely irregular. Example of text Blockquote Ram has taken an ...
2
votes
2answers
83 views

Why does GPT-2 Exclude the Transformer Encoder?

After looking into transformers, BERT, and GPT-2, from what I understand, GPT-2 essentially uses only the decoder part of the original transformer architecture and uses masked self-attention that can ...
0
votes
0answers
17 views

Why (not) using pre-processing before using Transformer models?

Regarding the use of pre-processing techniques before using Transformers models, I read this post that apparently says that these measures are not so necessary nor interfere so much in the final ...
0
votes
0answers
20 views

Text analysis or just statistical model?

For a college class, I have to train an ML model. I have an idea for a project but I am unsure if it is wise to attempt to solve this problem with ML as opposed to maybe more conventional methods of ...
0
votes
0answers
16 views

Extracting keywords from documents and predicting future keywords

I apologise for the newbie question! I've played with ML before, but at a very superficial level. I now have an interesting problem to solve and I wondered whether you could share some thoughts - I ...
2
votes
1answer
47 views

I have 5000 html files (structured text), how can I generate a new one that “resembles” those?

I don't know anything about ML or NLP, but I was asked by someone to create brand new statutes (written laws) that resemble the ones currently in effect in my country. I have already gathered the laws,...
0
votes
0answers
16 views

Adding BERT embeddings in BiLSTM embedding layer

I am want to use BERT embeddings in the BiLSTM embedding layer instead of Word2Vec or FastText Embeddings. There is any code to do that?
0
votes
1answer
33 views

How to train a sequence labeling model with annotations from three annotators?

I have a dataset of movie reviews annotated by 3 persons. The following example contains one sentence with corresponding annotations from 3 different persons. ...
1
vote
1answer
35 views

Can Facebook's LASER be used like BERT?

Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis? From my understanding, they created an embedding that allows for similar words in different languages ...
0
votes
2answers
83 views

Is my approach to building an RNN to predict the probability that the word is in English appropriate?

Goal To build an RNN which would receive a word as an input, and output the probability that the word is in English (or at least would be English sounding). Example ...
2
votes
3answers
111 views

Is there a relationship between Computer Algebra and NLP?

My intuition is that there is some overlap between understanding language and symbolic mathematics (e.g. algebra). The rules of algebra are somewhat like grammar, and the step-by-step arguments get ...
0
votes
1answer
35 views

Should I need to use BERT embeddings while tokenizing using BERT tokenizer?

I am new to BERT and NLP and I am a little confused with tokenization and word embedding. My doubt is if I use the BertTokenizer for tokenizing a sentence then do I have to compulsorily use ...

1
2 3 4 5
11