Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

How to Extract Specific Information From Web Pages Containing Job Postings?

I am currently working on my first research project, which is to implement machine learning based web scraping in the e-recruitement environment. Currently, I am building the dataset that needs to be ...
user avatar
-2 votes
0 answers
19 views

Converting word numerals to numbers

I am interested in the task of converting word numerals to numbers, e.g 'twenty two' -> 22 'hundred five fifteen eleven' -> 105 1511 etc. And the problem I can't understand at all currently is ...
user avatar
0 votes
2 answers
34 views

What is the best way to create a vector representation (with fasttext) of a list of words?

Basically what I want to do is to create a single vector representation of a list of skills belonging to employees at a company (one list per employee). The embedding will be a representation of an ...
user avatar
0 votes
1 answer
20 views

Can I use Sentence-Bert to embed event triples?

I extracted event triples from sentences using OpenIE. Can I concatenate the components in the event triple to make it a sentence and use Sentence-Bert to embed the event? It seems no one has done ...
user avatar
1 vote
1 answer
41 views

Current state of the art and datasets for combining NLP and CV?

I was considering a scenario where natural language processing (NLP) and computer vision (CV) are combined, for example in extended reality systems that get as input both natural language and non-...
user avatar
  • 121
2 votes
0 answers
38 views

Why use a fully connected layer for attention?

In the paper Neural Machine Translation by Jointly Learning to Align and Translate, attention is used with a single fully connected layer. Specifically, in the auto-regressive set up (equation 4), the ...
user avatar
  • 121
0 votes
1 answer
25 views

How to configure a neural network to selectively change only certain characters in a string?

I'm trying to figure out how to train a neural network to macronize Latin text. Essentially, in Latin, vowels can either be long or short, and length is indicated with a macronized character: i.e. o ...
user avatar
0 votes
0 answers
18 views

Can an advanced Bert-like model succeed where standard BERT failed?

I have a very difficult binary text classification problem for which I am trying to fine-tune a bert model (bert_en_uncased_L-12). By "difficult" I mean, the text data contains a large ...
user avatar
  • 11
0 votes
1 answer
43 views

Is there any way to train a regression model with negative values that is more stable?

I have a regression model where my target values contain roughly 60% negative values and 40% positive values. My model architecture includes a robert-large, 1 linear layer. I trained it after 1 epoch, ...
user avatar
1 vote
0 answers
22 views

What are semantic word spaces in NLP?

In the abstract of this paper, it's written Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. I would like to understand what semantic ...
user avatar
  • 121
3 votes
1 answer
37 views

Why do Convolution Neural Networks work on NLP/sequential tasks?

I have read some articles where people use 1D CNN for NLP tasks like sentiment analysis. My questions are, given that CNNs are largely used for images, how/why does this work on sequences/NLP tasks? ...
user avatar
0 votes
0 answers
13 views

Is there an unsupervised learning method for determine the most common questions within a dataset?

I have a dataset consisting of questions from customers. I am curious of the n most frequent asked questions, regardless of the variation the questions might appear in. Is there NLP methods for ...
user avatar
0 votes
0 answers
41 views

Can you train GPT-J to use a specific list of words and prioritise them?

Can you train GPT-J to use a specific list of words and prioritise them? If so, please could you share how I would go about this? Say you're using GPT-J to write a story, you might wish to mention ...
user avatar
1 vote
0 answers
19 views

Examples of self-explainable models used in NLP other than prototype-based

I am looking for all the methods used in NLP that are self-explainable, or explainable by design. That is to say, the ones that use the predictive model itself to explain the entire model's predictive ...
user avatar
0 votes
1 answer
49 views

BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it

I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word. I ...
user avatar
  • 101
1 vote
0 answers
26 views

Is there any research on anger and distrust detection (presence and level of political cynicism)?

The undergrad research project I'm working on would require me to detect presence and level of political cynicism from reddit posts. According to definition political cynicism consists of anger ...
user avatar
  • 11
0 votes
0 answers
12 views

Vector to sequence RNNs: do they take a random initial "prompt"?

I am going through the Deep Learning book by Ian Goodfellow (here) and came by the architecture for a vector to sequence RNN (Figure 10.9). I am not sure I understand how this architecture works and ...
user avatar
0 votes
2 answers
42 views

How to represent multi-label colours in one-hot encoding?

Say I want to predict the price of a gemstone based on its colour. I have two options: averaging over its colour on an RGB scale, or using its textual description. If I was to choose the latter, how ...
user avatar
0 votes
1 answer
27 views

How can I vectorize fictional single word (not sentence!) for classification?

I am working on fictional single words (names) generator that have to sound like words from a given sample. I have the generator up and running that gives reasonable words 70% of time. I thought of ...
user avatar
  • 101
0 votes
0 answers
20 views

How do I perform automatic evaluation of my NLP model?

I have a model which converts sets of keywords to sentence, but I've to quantify it's quality. In computer vision, we would calculate the model's accuracy, I'm kind of lost and how do I go about using ...
user avatar
1 vote
0 answers
16 views

When training a seq2seq model is it better to train using the models outputs or expected outputs?

When training any seq2seq model you have a target and a source. The source may be a sentence ...
user avatar
  • 1,344
0 votes
0 answers
66 views

Is attention always better then an RNN/CNN?

We've all read the attention is all you need paper, but is it really all you need? Can you effectively replace any RNN/CNN with an attention transformer and see better results?
user avatar
  • 1,344
0 votes
0 answers
16 views

Convolutional network for multilabel classification in NLP

I am trying to label code snippets and I base on this article: https://arxiv.org/pdf/1906.01032.pdf My dataset is just code snippets (tokenized as ascii characters) and 500 different labels from ...
user avatar
0 votes
0 answers
9 views

What are confounding information, things that could also explain the outcome in a performance attribution model?

I am reading a paper about a performance attribution method, a method for finding the degree to which an outcome can be attributed to parts of a text while controlling for potential confounders. I ...
user avatar
0 votes
1 answer
47 views

Does it make sense to add an additional attention layer while fine-tuning Bert?

I'm fine tuning a Bert/Roberta model for a classification task. As I need to improve my results, I'm thinking about to add an additional attention layer after Bert model and before dense and dropout ...
user avatar
0 votes
0 answers
19 views

Where Can I Find Resources on Extracting Meaningful Content From Web Pages?

I am in the process of conducting a literature review for my thesis. Currently, I am struggling when it comes to developing a theoretical framework/methodology or to even correctly outline an approach ...
user avatar
0 votes
0 answers
12 views

How to interpret integrated gradients in an NLP toxic text classification use-case?

I am trying to understand how integrated gradients work in the NLP case. Let $F: \mathbb{R}^{n} \rightarrow[0,1]$ a function representing a neural network, $x \in \mathbb{R}^{n}$ an input and $x' \in ...
user avatar
0 votes
0 answers
17 views

Which NLP methods use gradient and activation methods?

I am doing a literature review of gradient-based methods for NLP. Yet, apart from linear and logistic regression, I have little knowledge of other methods using the gradient. So, I have no knowledge ...
user avatar
0 votes
1 answer
26 views

What are examples of simple gradient based NLP models?

I am looking for a simple example of gradient-based methods for NLP. More specifically I am looking for post-hoc local explanations gradient-based methods, that is to say, which explain a single ...
user avatar
0 votes
0 answers
9 views

How to get match probability of the input text to the historical data?

I have 4 bln rows of data and unique 7500 products. Product ID has the commercial name of the product, which means that Product X can have a different set of descriptions. For example, ...
user avatar
3 votes
1 answer
482 views

Why are GAN models not heavily used for NLP?

I am wondering why there has not been more usage of GANs for NLP. I know there has been research on the subject (The Google Scholar page for the subject is here). Are there any specific reasons why ...
user avatar
1 vote
1 answer
28 views

ontological tree for the concept of a word (eg "chair")

I am novice to AI, but I would like to learn the general idea that a machine understands the concepts in a text document. I would like to ask wether there is an ontological tree of concepts, like a ...
user avatar
  • 13
0 votes
0 answers
30 views

Modify the architecture of the pre-trained model

I'm trying to remove some layers and reduce the size of fully connected in pre-trained BERT model. And then I'm going to fine-tune the modified models on GLUE. But the problem is that the modified ...
user avatar
  • 1
2 votes
1 answer
72 views

How might AI analyze abusive discussion using natural language grammar?

Opening thoughts This does not only apply to SE comments, but the idea in general. This is not a Question for Linguistics.SE; those Questions might come later, after AI analysis. Example Linguistics ...
user avatar
1 vote
1 answer
22 views

Are any AI systems available, or in development, for finding and analysing fallacious inference in natural language text?

Poor reasoning, and ignorance in general, is the source of a lot of suffering and evil. Covertly erroneous logic is often used in manipulation. And much of this broken thought is being used directly ...
user avatar
  • 136
0 votes
0 answers
19 views

How to interpret Transformer output

In this (https://towardsdatascience.com/a-detailed-guide-to-pytorchs-nn-transformer-module-c80afbc9ffb1) article the author says, that the output of the ...
user avatar
1 vote
2 answers
34 views

Is there any reason for giving an index to a token based on its frequency in the text?

In pre-processing of text, we need to assign a number for each token in a text. Then only we can pass it to a model. In pre-processing of text, we need to assign a number for each token in a text. The ...
user avatar
  • 3,099
0 votes
0 answers
120 views

Next Sentence Prediction for 5 sentences using BERT

I am given a dataset in which each instance consisting of 5 sentences. The goal is to predict the sequence of numbers which represent the order of these sentences. For example, given a story: He went ...
user avatar
0 votes
1 answer
87 views

Dimensions of a Transformer model and purpose of masking [closed]

I'm currently studying the Transformer model (Attention is all you need) and after reading it I still have some questions for which I get conflicting answers if I google them: What exactly are the ...
user avatar
0 votes
0 answers
181 views

How do we reduce the output dimensions of BERT?

The output dimensions of BERT are 768-dimensional, is it possible to reduce them to a lower, custom number? For example, if there is another BERT-based transformer model which has a lower count of ...
user avatar
0 votes
1 answer
23 views

Are positional embeddings computed during or before training?

I'm trying to practically frame the concept of positional embeddings as introduced in the original paper. As far as I've understood, what we do is basically creating some other vectors in addition to ...
user avatar
0 votes
0 answers
18 views

Variational AutoEncoders | Is Latent space an Embedding space? [duplicate]

I am learning about Variational Autoencoders and it is mentioned that the objective of an encoder is to produce a latent space, "encoding vector". Question: Is latent space just an "...
user avatar
  • 135
0 votes
1 answer
210 views

Compare Strings composed from 2-3 words using NLP/ML(Python)

I have a database of books. Each book have a list of categories that describe the genre/topics of the book (I use Python models). Most of the time, the categories in the list are composed from 1-3 ...
user avatar
0 votes
0 answers
53 views

Which deep learning architecture for NLP is most accurate (and at the same time easy to implement)?

I have created a data set with 30.000 text documents (each text file is rather small with respect to its length), which are labelled with 0 and 1. Using this data set, I want to train a deep learning ...
user avatar
1 vote
1 answer
182 views

Sequence Embedding using embedding layer: how does the network architecture influence it?

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
user avatar
0 votes
0 answers
21 views

model to generate suggestions for improving the cosine similarity of two documents?

I am working on a system that compares a source document to a target document and then generate alternative variations of the source document. The goal is to reach a higher cosine similarity between ...
user avatar
0 votes
0 answers
43 views

LSTM performance strictly decreases with sequence length input

I'm working on an event binary classification problem. More specifically, for a given event E I know some info about the event itself just before it's supposed to happen, encoded in an embedding ...
user avatar
0 votes
0 answers
12 views

How to use tabular data and figures when the structure of data varies?

Suppose I have several journal articles. I would like to train a binary classifier on whether the journal article is insightful. NLP models such as BERT certainly fit my need by scanning the whole ...
user avatar
0 votes
1 answer
97 views

Which data representation of text as input for NLP Deep Learning models?

I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. ...
user avatar
0 votes
1 answer
125 views

Generating automatic sports commentary (NLG)

I am trying to develop a "simple" announcer for sports segments that mainly consists of events like goals, fouls, substitutions, and many other events that could happen in many sports. The ...
user avatar

1
2 3 4 5
12