Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Is there any neural network model that can perform multiple NLP steps at once?

I realize most NLP algorithms have multiple steps. (e.g. OCR/speech rec > syntax > semantics > response logic > semantic output > natural language output) Is there any NN model that can ...
1
vote
0answers
14 views

What is the best approach for sentiment analysis when the text is very brief?

I'm working on a project to do sentiment analysis but my data is not long and properly formatted text. It's more likely to be very short sentences, e.g. tweets (in full tweet lingo), quick reviews of ...
0
votes
1answer
27 views

What dataset might Elon Musk's Dall-E have used?

When I saw the rollout of Dall-E, it can generate many imaginative images from the description, even some peculiar images, but I could not figure out how did they actually create this kind of dataset ...
0
votes
0answers
15 views

Text generation with LSTM with multiple correlated inputs

I am currently working on a music-generation project, inspired by an already existing project called Deepbach. My dataset are the Bach chorales, which are all composed of 4 independent (but related) ...
1
vote
0answers
19 views

Python catch the reference between 2 strings [closed]

I am developing a simple personal assistant using python. It takes in verbal input into a string, processes it and gives an output. For e.g. when the user says - Show me pictures of Robert Downey ...
1
vote
1answer
49 views

How to extract parameters from a text using AI/NLP

lets say I have three texts: "make a heading that says hello word" "make a heading of hello world" "create heading consist of hello world" How can I fetch those groups ...
2
votes
1answer
41 views

How can I find words in a string that are related to a given word, then associate a sentiment to that found word?

I came up with an NLP-related problem where I have a list of words and a string. My goal is to find any word in the list of words that is related to the given string. Here is an example. Suppose a ...
0
votes
0answers
21 views

Is there a reference that describes Recurrent Neural Networks for NLP tasks?

I would like some references of works that try to understand the functioning of any kind of RNN in natural language processing tasks. They can be any work that tries to explain the functioning of the ...
0
votes
1answer
22 views

How to understand 'losses' in Spacy's custom NER training engine?

From the tid-bits, I understand of neural networks (NN), the Loss function is the difference between predicted output and expected output of the NN. I am following this tutorial, the losses are ...
2
votes
0answers
25 views

T5 or BERT for sentence correction/generation task?

I have sentences with some grammatical errors , with no punctuations and digits written in words... something like below: As you can observe, a proper noun , winston isnt highlighted with capital in ...
0
votes
0answers
6 views

What is this for loop doing in custom NER code in Spacy and what is an 'annotation' in Spacy?

I am writing a code to train custom entities in Spacy's NER engine. I am stuck in understanding small part of code from an online tutorial. Here's a link to the tutorial. The following code is line ...
0
votes
1answer
22 views

Should we use a pre-trained model or a blank model for custom entity training of NER in spacy?

Further to my last question, I am training a custom entity of FOODITEM to be recognized by Spacy's Name Entity Recognition engine. I am following tutorials online, following is the advise given in ...
1
vote
0answers
18 views

Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?

I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
0
votes
0answers
12 views

Time series analysis using computer vision principles

I'm just starting to explore topics within computer vision and curious if there are any concepts in that area that could be applied to segmenting multivariate time series with the goal of grouping ...
0
votes
1answer
36 views

How to design a NLP algorithm to find a food item in menu card list?

I am new to NLP and AI in general. I am just expecting springboard information so that I can skip all the introduction to NLP websites. I have just started studying NLP and want to know how to go ...
0
votes
0answers
44 views

Which approach should I follow to classify tweets into topics?

I am approaching machine learning for the first time because of my studies. I have been given a bunch of tweets and the goal is to classify them per topic. The topics are not predefined, but need to ...
1
vote
0answers
21 views

Building a resume recommendation for a job post?

There are few challenges I am facing when building a resume recommendation for a particular job positing. Let's say we convert the resume into a vector on n-dimensions and job description also as an n-...
0
votes
1answer
49 views

Are there any good alternatives to an LSTM language model for text generation?

I have a trained LSTM language model and want to use it to generate text. The standard approach for this seems to be: Apply softmax function Take a weighted random choice to determine the next word ...
0
votes
1answer
26 views

Can the attention mechanism improve the performance in the case of short sequences?

I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise. ...
0
votes
0answers
12 views

Chatbot with Context Management and Awareness

I am currently implementing a closed-domain FAQ chatbot using https://www.sbert.net/index.html as my main model for answering questions from the users. I wish to extend the functionality of the ...
1
vote
0answers
50 views

How to handle long sequences with transformers?

I have a time series sequence with 10 million steps. In step $t$, I have a 400 dimensional feature vector $X_t$ and a scalar value $y_t$ which I want to predict during inference time and I know during ...
2
votes
1answer
42 views

What is different in each head of a multi-head attention mechanism?

I have a difficult time understanding the "multi-head" notion in the original transformer paper. What makes the learning in each head unique? Why doesn't the neural network learn the same ...
1
vote
0answers
4 views

POS Tag Frequency and Language Translation

When we translate a text from one language to another, how does the frequency of various POS tags change? So let's say we have a text in English with 10% nouns, 20% adjectives, 15% adverbs, 25% verbs, ...
1
vote
0answers
38 views

What is the gradient of an attention unit?

The paper Attention Is All You Need describes the Transformer architecture, which describes attention as a function of the queries $Q = x W^Q$, keys $K = x W^K$, and values $V = x W^V$: $\text{...
2
votes
1answer
44 views

What is the cost function of a transformer?

The paper Attention Is All You Need describes the transformer architecture that has an encoder and a decoder. However, I wasn't clear on what the cost function to minimize is for such an architecture. ...
0
votes
0answers
13 views

Finding or creating a dataset for Neural Text Simplification

I'm currently starting a research project focused on NLP. One of the steps involved in this project will be the development of a text simplification system, probably using a neural encoder-decoder ...
0
votes
0answers
21 views

Is it possible to automatically remove ignore (or remove) the equations (and other noisy elements) while performing OCR?

I have academic pdf data. I am using OCR for converting it into text format. The pdf has a few mathematical equations and terms which are acting as noise for my task. Any way through which the task of ...
2
votes
1answer
31 views

Transformers: how to get the output (keys and values) of the encoder?

I was reading the paper Attention Is All You Need. It seems like the last step of the encoder is a LayerNorm(relu(WX + B) + X), i.e. an add + normalization. This should result in a $n$ x $d^{model}$ ...
2
votes
1answer
27 views

Transformers: how does the decoder final layer output the desired token?

In the paper Attention Is All You Need, this section confuses me: In our model, we share the same weight matrix between the two embedding layers [in the encoding section] and the pre-softmax linear ...
2
votes
0answers
34 views

Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?

When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN. Lets say instead of using dense ...
0
votes
1answer
52 views

Making generated texts from “data-to-text” more variable

I am diving in data-to-text generation for long articles (> 1000 words). After creating a template and fill it with data I am currently going down on paragraph level and adding different paragraphs,...
2
votes
1answer
32 views

Predict next event based on previous events and discrete reward values

Suppose, I have several sequences that include a series of text (the length of sequence can be varied). Also, I have some related reward value. however, the value is not continuous like the text. It ...
0
votes
0answers
12 views

What Deep Learning Applications Might Require Super-Computers or “SuperPODs”

With the release of NVIDIA's DGX SuperPOD of A100 GPUs, supercomputers will/are becoming more and more common-place. What potential deep learning tasks/applications might become more accessible with ...
0
votes
0answers
8 views

Best strategy for Classification of Science Subjects. Phy, Chem , Maths and Bio? BERT, Transformers, Attention+SLTM, Self-Attention+LSTM?

I am working on a project where I have to first classify the Subjects of the given question and then the respective Chapter and then the sub-topic. In a nutshell, I have to predict the Subject, Grade ...
0
votes
0answers
10 views

How to make specific test data prediction with fitted GaussianNB Classifier in Python

I'm trying to make news classification. Here is the neural network: ...
1
vote
1answer
278 views

How is Google Translate able to convert texts of different lengths?

According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?
2
votes
1answer
73 views

Are there any meaningful books entirely written by an artificial intelligence?

Are there any meaningful books entirely written by an artificial intelligence? I mean something with meaning, unlike random words or empty books. Something that can be charactersed as fiction ...
1
vote
1answer
29 views

Is there a way to provide multiple masks to BERT in MLM task?

I'm facing a situation where I've to fetch probabilities from BERT MLM for multiple words in a single sentence. ...
0
votes
0answers
15 views

How would named recognition and entity linking interface with subsequent machine learning model?

I have a database containing summary of the movie, the price of the movie, location, language. I would like to make a prediction on 'like' based on previous like of a user. I am wonder how the output ...
1
vote
1answer
46 views

Are FSA and FSTs used in NLP nowadays?

Finite state automata and transducers are computational models that were widely used decades before in natural language processing for morphological parsing and other nlp tasks. I wonder if these ...
0
votes
0answers
19 views

Extracting specific information from an Invoice images

Tried to extract only specific information from the images but Couldn't, We have to automate this process using this as the format of the Images keeps on changing. LinkSample data What I have Tried: <...
1
vote
1answer
53 views

The mathematics in the CBOW and Skip-Gram models

this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations ...
0
votes
0answers
16 views

What BERT predicts when the token supposed to be masked is not masked?

I am reading the BERT paper. In the paper, they say that: Although this allows us to obtain a bidirec- tional pre-trained model, a downside is that we are creating a mismatch between pre-training and ...
1
vote
0answers
17 views

Bechmark models for Text Classification / Sentiment Classification

I am currently working on a novel application in NLP where I try to classify empathic and non-empathic texts. I would like to compare the performance of my model to some benchmark models. As I am ...
2
votes
0answers
21 views

What is the difference between text-based image retrieval and natural language object retrieval?

I'm working on creating a model that locates the object in the scene (2D image or 3D scene) using a natural language query. I came across this paper on natural language object retrieval, which ...
0
votes
0answers
18 views

Transformer Language model produces only <pad> tokens when generating new sentences

I am training a word-level language model using the transformer module available in Pytorch. I am getting a really good training loss and the model is able to reproduce the sentences in the training ...
1
vote
0answers
38 views

Determining if an entity in free text is 'present' or 'absent'; what is this called in NLP?

I'm processing a semi-structured scientific document and trying to extract some specific concepts. I've actually made quite good progress without machine-learning so far, but I got to a block of true ...
0
votes
1answer
44 views

What is the best algorithm to solve the regression problem of predicting the number of languages a Wikipedia article can be translated to?

I'm doing a student project where I construct a model predicting the number of languages that a given Wikipedia article is translated into (for example, the article TOYOTA is translated into 93 ...
2
votes
1answer
48 views

Any comparison between transformer and RNN+Attention on the same dataset?

I am wondering what is believed to be the reason for superiority of transformer? I see that some people believe because of the attention mechanism used, it’s able to capture much longer dependencies. ...
5
votes
2answers
134 views

What is the purpose of Decoder mask (triangular mask) in Transformer?

I'm trying to implement transformer model using this tutorial. In the decoder block of the Transformer model, a mask is passed to "pad and mask future tokens in the input received by the decoder&...

1
2 3 4 5
10