Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
14 views

Splitting a sequence into subsequences [closed]

I am working on an NLP problem with dependencies. In which I wish to break my sentence into sub phrases according to their dependencies. For example : text = Day without sunshine is like night. dep = ...
user avatar
2 votes
1 answer
28 views

Why do Convolution Neural Networks work on NLP/sequential tasks?

I have read some articles where people use 1D CNN for NLP tasks like sentiment analysis. My questions are, given that CNNs are largely used for images, how/why does this work on sequences/NLP tasks? ...
user avatar
0 votes
0 answers
9 views

Is there an unsupervised learning method for determine the most common questions within a dataset?

I have a dataset consisting of questions from customers. I am curious of the n most frequent asked questions, regardless of the variation the questions might appear in. Is there NLP methods for ...
user avatar
0 votes
0 answers
22 views

Can you train GPT-J to use a specific list of words and prioritise them?

Can you train GPT-J to use a specific list of words and prioritise them? If so, please could you share how I would go about this? Say you're using GPT-J to write a story, you might wish to mention ...
user avatar
1 vote
0 answers
14 views

Examples of self-explainable models used in NLP other than prototype-based

I am looking for all the methods used in NLP that are self-explainable, or explainable by design. That is to say, the ones that use the predictive model itself to explain the entire model's predictive ...
user avatar
0 votes
1 answer
35 views

BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it

I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word. I ...
user avatar
  • 101
0 votes
0 answers
22 views

Is there any research on anger and distrust detection (presence and level of political cynicism)?

The undergrad research project I'm working on would require me to detect presence and level of political cynicism from reddit posts. According to definition political cynicism consists of anger ...
user avatar
0 votes
0 answers
11 views

Vector to sequence RNNs: do they take a random initial "prompt"?

I am going through the Deep Learning book by Ian Goodfellow (here) and came by the architecture for a vector to sequence RNN (Figure 10.9). I am not sure I understand how this architecture works and ...
user avatar
0 votes
2 answers
42 views

How to represent multi-label colours in one-hot encoding?

Say I want to predict the price of a gemstone based on its colour. I have two options: averaging over its colour on an RGB scale, or using its textual description. If I was to choose the latter, how ...
user avatar
0 votes
1 answer
27 views

How can I vectorize fictional single word (not sentence!) for classification?

I am working on fictional single words (names) generator that have to sound like words from a given sample. I have the generator up and running that gives reasonable words 70% of time. I thought of ...
user avatar
  • 101
0 votes
0 answers
18 views

How do I perform automatic evaluation of my NLP model?

I have a model which converts sets of keywords to sentence, but I've to quantify it's quality. In computer vision, we would calculate the model's accuracy, I'm kind of lost and how do I go about using ...
user avatar
1 vote
0 answers
13 views

When training a seq2seq model is it better to train using the models outputs or expected outputs?

When training any seq2seq model you have a target and a source. The source may be a sentence ...
user avatar
  • 1,324
0 votes
0 answers
28 views

Is attention always better then an RNN/CNN?

We've all read the attention is all you need paper, but is it really all you need? Can you effectively replace any RNN/CNN with an attention transformer and see better results?
user avatar
  • 1,324
0 votes
0 answers
13 views

Convolutional network for multilabel classification in NLP

I am trying to label code snippets and I base on this article: https://arxiv.org/pdf/1906.01032.pdf My dataset is just code snippets (tokenized as ascii characters) and 500 different labels from ...
user avatar
0 votes
0 answers
8 views

What are confounding information, things that could also explain the outcome in a performance attribution model?

I am reading a paper about a performance attribution method, a method for finding the degree to which an outcome can be attributed to parts of a text while controlling for potential confounders. I ...
user avatar
0 votes
1 answer
25 views

Does make sense to add an additional Attention layer while Fine-Tuning Bert?

I'm fine tuning a Bert/Roberta model for a classification task. As I need to improve my results I'm thinking about to add an additional Attention layer after Bert Model and before Dense and Dropout ...
user avatar
0 votes
0 answers
19 views

Where Can I Find Resources on Extracting Meaningful Content From Web Pages?

I am in the process of conducting a literature review for my thesis. Currently, I am struggling when it comes to developing a theoretical framework/methodology or to even correctly outline an approach ...
user avatar
0 votes
0 answers
9 views

How to interpret integrated gradients in an NLP toxic text classification use-case?

I am trying to understand how integrated gradients work in the NLP case. Let $F: \mathbb{R}^{n} \rightarrow[0,1]$ a function representing a neural network, $x \in \mathbb{R}^{n}$ an input and $x' \in ...
user avatar
0 votes
0 answers
16 views

Which NLP methods use gradient and activation methods?

I am doing a literature review of gradient-based methods for NLP. Yet, apart from linear and logistic regression, I have little knowledge of other methods using the gradient. So, I have no knowledge ...
user avatar
0 votes
1 answer
24 views

What are examples of simple gradient based NLP models?

I am looking for a simple example of gradient-based methods for NLP. More specifically I am looking for post-hoc local explanations gradient-based methods, that is to say, which explain a single ...
user avatar
0 votes
0 answers
9 views

How to get match probability of the input text to the historical data?

I have 4 bln rows of data and unique 7500 products. Product ID has the commercial name of the product, which means that Product X can have a different set of descriptions. For example, ...
user avatar
3 votes
1 answer
145 views

Why are GAN models not heavily used for NLP?

I am wondering why there has not been more usage of GANs for NLP. I know there has been research on the subject (The Google Scholar page for the subject is here). Are there any specific reasons why ...
user avatar
1 vote
1 answer
28 views

ontological tree for the concept of a word (eg "chair")

I am novice to AI, but I would like to learn the general idea that a machine understands the concepts in a text document. I would like to ask wether there is an ontological tree of concepts, like a ...
user avatar
  • 13
0 votes
0 answers
21 views

Modify the architecture of the pre-trained model

I'm trying to remove some layers and reduce the size of fully connected in pre-trained BERT model. And then I'm going to fine-tune the modified models on GLUE. But the problem is that the modified ...
user avatar
  • 1
2 votes
1 answer
70 views

How might AI analyze abusive discussion using natural language grammar?

Opening thoughts This does not only apply to SE comments, but the idea in general. This is not a Question for Linguistics.SE; those Questions might come later, after AI analysis. Example Linguistics ...
user avatar
1 vote
1 answer
22 views

Are any AI systems available, or in development, for finding and analysing fallacious inference in natural language text?

Poor reasoning, and ignorance in general, is the source of a lot of suffering and evil. Covertly erroneous logic is often used in manipulation. And much of this broken thought is being used directly ...
user avatar
  • 136
0 votes
0 answers
13 views

How to interpret Transformer output

In this (https://towardsdatascience.com/a-detailed-guide-to-pytorchs-nn-transformer-module-c80afbc9ffb1) article the author says, that the output of the ...
user avatar
1 vote
2 answers
33 views

Is there any reason for giving an index to a token based on its frequency in the text?

In pre-processing of text, we need to assign a number for each token in a text. Then only we can pass it to a model. In pre-processing of text, we need to assign a number for each token in a text. The ...
user avatar
  • 2,977
0 votes
0 answers
54 views

Next Sentence Prediction for 5 sentences using BERT

I am given a dataset in which each instance consisting of 5 sentences. The goal is to predict the sequence of numbers which represent the order of these sentences. For example, given a story: He went ...
user avatar
0 votes
1 answer
60 views

Dimensions of a Transformer model and purpose of masking [closed]

I'm currently studying the Transformer model (Attention is all you need) and after reading it I still have some questions for which I get conflicting answers if I google them: What exactly are the ...
user avatar
0 votes
0 answers
113 views

How do we reduce the output dimensions of BERT?

The output dimensions of BERT are 768-dimensional, is it possible to reduce them to a lower, custom number? For example, if there is another BERT-based transformer model which has a lower count of ...
user avatar
0 votes
1 answer
21 views

Are positional embeddings computed during or before training?

I'm trying to practically frame the concept of positional embeddings as introduced in the original paper. As far as I've understood, what we do is basically creating some other vectors in addition to ...
user avatar
0 votes
0 answers
17 views

Variational AutoEncoders | Is Latent space an Embedding space? [duplicate]

I am learning about Variational Autoencoders and it is mentioned that the objective of an encoder is to produce a latent space, "encoding vector". Question: Is latent space just an "...
user avatar
  • 135
0 votes
0 answers
105 views

Compare Strings composed from 2-3 words using NLP/ML(Python)

I have a database of books. Each book have a list of categories that describe the genre/topics of the book (I use Python models). Most of the time, the categories in the list are composed from 1-3 ...
user avatar
0 votes
0 answers
41 views

Which deep learning architecture for NLP is most accurate (and at the same time easy to implement)?

I have created a data set with 30.000 text documents (each text file is rather small with respect to its length), which are labelled with 0 and 1. Using this data set, I want to train a deep learning ...
user avatar
1 vote
1 answer
99 views

Sequence Embedding using embedding layer: how does the network architecture influence it? [closed]

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
user avatar
0 votes
0 answers
21 views

model to generate suggestions for improving the cosine similarity of two documents?

I am working on a system that compares a source document to a target document and then generate alternative variations of the source document. The goal is to reach a higher cosine similarity between ...
user avatar
0 votes
0 answers
36 views

LSTM performance strictly decreases with sequence length input

I'm working on an event binary classification problem. More specifically, for a given event E I know some info about the event itself just before it's supposed to happen, encoded in an embedding ...
user avatar
0 votes
0 answers
12 views

How to use tabular data and figures when the structure of data varies?

Suppose I have several journal articles. I would like to train a binary classifier on whether the journal article is insightful. NLP models such as BERT certainly fit my need by scanning the whole ...
user avatar
0 votes
1 answer
81 views

Which data representation of text as input for NLP Deep Learning models?

I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. ...
user avatar
0 votes
1 answer
105 views

Generating automatic sports commentary (NLG)

I am trying to develop a "simple" announcer for sports segments that mainly consists of events like goals, fouls, substitutions, and many other events that could happen in many sports. The ...
user avatar
0 votes
0 answers
13 views

XLMRoberta loss remains constant over iterations for TokenClassification task

I have created a simple XLMRoberta model for token classification. The task is to predict the quality of translation for each token/word. The data looks something like this, where the first sentence ...
user avatar
0 votes
0 answers
26 views

What are the types of inputs used for RNN in literature given sentences?

Suppose there are $m$ sentences in a text file and the number of distinct words is equal to $n$. The goal is to get word embeddings using RNN. We know that it is impossible to pass any word, which is ...
user avatar
  • 2,977
1 vote
1 answer
36 views

Is image machine translation done in two steps?

Suppose I have images of hand-written Japanese text. If I want to translate those images, would my ML algorithm be a 2-step model (for example, a CNN to convert the image into Japanese characters/...
user avatar
0 votes
0 answers
37 views

Are there any inference memory requirement tables for Hugging Face transformers?

Hugging Face has a very large list of supported transformers. They provide a table which gives the status on whether or not a transformer has a slow tokenizer, a fast tokenizer, PyTorch support, ...
user avatar
1 vote
1 answer
100 views

Which pre-processing steps are necessary for Deep Learning models to solve a document classification problem?

I have created a data set with 30.000 text documents (each text file is rather small with respect to its length), which are labelled with 0 and 1. Using this data set, I want to train machine learning ...
user avatar
0 votes
0 answers
50 views

Fine tuning BERT for token level classification

I want to try self-supervised and semi-supervised learning for my task, which relates to token-wise classification for the 2 sequences of sentences (source and translated text). The labels would be ...
user avatar
1 vote
1 answer
34 views

What kind of NN to use to find misprints in test

I have a bunch of unique full names of users. I made pseudo-physical model to emulate misprints of desktop and mobile users (hence, fatfingering, jumpy fingers, accidentals touches of touch bar etc.) ...
user avatar
0 votes
0 answers
16 views

Is there a way to train Doc2Vec on a corpus of docs and be able to take a novel doc and see how similar it is to the training corpus?

I have a project idea, where I train a bunch of documents on Doc2Vec, and then take a novel, input doc, and ideally be able to be told how similar it is to the docs supplied for training as a whole or ...
user avatar
  • 131
0 votes
0 answers
23 views

How to find the accuracy of LDA model?

I am working on topic modeling using the latent Dirichlet allocation model. I have a dataset that contains tweets and topics corresponding to these tweets. In total, there are 65 different topics. I ...
user avatar

1
2 3 4 5
12