Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Can you add more layers in Transformers for classification

I am working on NLP classification task and I am using the AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=5) method. ...
2
votes
0answers
25 views

What is the difference between zero-padding and character-padding in Recurrent Neural Networks?

For RNN's to work efficiently, we vectorize the operations, which results in an input matrix of shape (m, max_seq_len) where m ...
1
vote
1answer
38 views

How are certain machine learning models able to produce variable-length outputs given variable-length inputs?

Most machine learning models, such as multilayer perceptrons, require a fixed-length input and output, but generative (pre-trained) transformers can produce sentences or full articles of variable ...
1
vote
0answers
13 views

Adding corpus to BERT for QA

I was wondering about SciBERT's QA abilities using SQuAD. I have a scarce textual dataset consisting of less than 100 files where doctors are discussing cancer in dialogues. I want to add it to ...
0
votes
0answers
10 views

Text Analysis for Logical Consistency [closed]

Consider a “Transportation Law” of an specific country. And we know that passing laws are jobs of legislators and is “alive”. My questions are: (1) Is there an application software to assess such ...
0
votes
1answer
20 views

Are the Word2Vec encoding available online [closed]

I am trying to do an NLP project and was wondering if there is anywhere online where the Word2Vec encoding are stored. I want to search up a word and see what its encoding is. I have tried looking but ...
0
votes
1answer
23 views

NLP: Are hashtags tokenised?

I am exploring a potential NLP project and was wondering what generally is done with the hashtags words (e.g. #hello). Are those words ignored? is the # removed and the word tokenised? is it tokenised ...
1
vote
1answer
31 views

How to keep track of the subject/entity in a sentence?

I'm working on Sentiment Analysis, using HuggingFace to perform sentiment analysis on articles ...
2
votes
1answer
46 views

How would one disambiguate between two meanings of the same word in a sentence?

The boy lifted the bat and hit the ball. In the above sentence, the noun "bat" means the wooden stick. It does not mean bat, the flying mammal, which is also a noun. Using NLP libraries to ...
1
vote
0answers
24 views

What does the outputlayer of BERT for masked language modelling look like?

In the tutorial BERT – State of the Art Language Model for NLP the masked language modeling pre-training steps are described as follows: In technical terms, the prediction of the output words ...
1
vote
2answers
51 views

What kind of word embedding is used in the original transformer?

I am currently trying to understand transformers. To start, I read Attention Is All You Need and also this tutorial. What makes me wonder is the word embedding used in the model. Is word2vec or GloVe ...
0
votes
0answers
33 views

Is there a reason why no one combines word embeddings with the median?

Could you combine word embeddings with the median per dimension to get a document embedding? In my case I have a huge amount of words to build one document, which in turn should describe a topic. I ...
1
vote
0answers
14 views

How to scrape product data on supplier websites?

I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
-1
votes
0answers
14 views

sequential autoencoder: RNN decoding comparing to RNN with attention decoding

I've implemented an RNN-ecnoder-decoder model and the same model with attention mechanism. That is the RNN model: and that is the attentioned model: then I asked for a theoretical question: The ...
0
votes
0answers
28 views

Is using a LSTM, CNN or any other neural network model on top of a Transformer(using hidden states) overkill?

I have recently come across transformers, I am new to Deep Learning. I have seen a paper using CNN and BiLSTM on top of a transformer, the paper uses a transformer(XLM-R) for sentiment analysis in ...
0
votes
1answer
41 views

Is the working of RNNs, LSTM and GRU sequential or parallel?

You take any blog or any example and all they tell you about is the given picture below. It has 4 different matrices and 3 of whose weights are shared. So, I'm wondering how is this achieved in ...
0
votes
0answers
15 views

Is there any neural network model that can perform multiple NLP steps at once?

I realize most NLP algorithms have multiple steps. (e.g. OCR/speech rec > syntax > semantics > response logic > semantic output > natural language output) Is there any NN model that can ...
1
vote
0answers
21 views

What is the best approach for sentiment analysis when the text is very brief?

I'm working on a project to do sentiment analysis but my data is not long and properly formatted text. It's more likely to be very short sentences, e.g. tweets (in full tweet lingo), quick reviews of ...
1
vote
1answer
55 views

What dataset might Elon Musk's Dall-E have used?

Dall-E, it can generate many imaginative images from the description, even some peculiar images, how did they actually create this kind of dataset to train this AI , because there is not much of that ...
0
votes
0answers
15 views

Text generation with LSTM with multiple correlated inputs

I am currently working on a music-generation project, inspired by an already existing project called Deepbach. My dataset are the Bach chorales, which are all composed of 4 independent (but related) ...
1
vote
2answers
70 views

How to extract parameters from a text using AI/NLP

lets say I have three texts: "make a heading that says hello word" "make a heading of hello world" "create heading consist of hello world" How can I fetch those groups ...
2
votes
1answer
47 views

How can I find words in a string that are related to a given word, then associate a sentiment to that found word?

I came up with an NLP-related problem where I have a list of words and a string. My goal is to find any word in the list of words that is related to the given string. Here is an example. Suppose a ...
0
votes
0answers
23 views

Is there a reference that describes Recurrent Neural Networks for NLP tasks?

I would like some references of works that try to understand the functioning of any kind of RNN in natural language processing tasks. They can be any work that tries to explain the functioning of the ...
0
votes
1answer
75 views

How to understand 'losses' in Spacy's custom NER training engine?

From the tid-bits, I understand of neural networks (NN), the Loss function is the difference between predicted output and expected output of the NN. I am following this tutorial, the losses are ...
2
votes
0answers
32 views

T5 or BERT for sentence correction/generation task?

I have sentences with some grammatical errors , with no punctuations and digits written in words... something like below: As you can observe, a proper noun , winston isnt highlighted with capital in ...
0
votes
0answers
6 views

What is this for loop doing in custom NER code in Spacy and what is an 'annotation' in Spacy?

I am writing a code to train custom entities in Spacy's NER engine. I am stuck in understanding small part of code from an online tutorial. Here's a link to the tutorial. The following code is line ...
0
votes
1answer
22 views

Should we use a pre-trained model or a blank model for custom entity training of NER in spacy?

Further to my last question, I am training a custom entity of FOODITEM to be recognized by Spacy's Name Entity Recognition engine. I am following tutorials online, following is the advise given in ...
1
vote
0answers
21 views

Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?

I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
0
votes
0answers
24 views

Time series analysis using computer vision principles

I'm just starting to explore topics within computer vision and curious if there are any concepts in that area that could be applied to segmenting multivariate time series with the goal of grouping ...
0
votes
1answer
41 views

How to design a NLP algorithm to find a food item in menu card list?

I am new to NLP and AI in general. I am just expecting springboard information so that I can skip all the introduction to NLP websites. I have just started studying NLP and want to know how to go ...
1
vote
0answers
22 views

Building a resume recommendation for a job post?

There are few challenges I am facing when building a resume recommendation for a particular job positing. Let's say we convert the resume into a vector on n-dimensions and job description also as an n-...
0
votes
1answer
53 views

Are there any good alternatives to an LSTM language model for text generation?

I have a trained LSTM language model and want to use it to generate text. The standard approach for this seems to be: Apply softmax function Take a weighted random choice to determine the next word ...
0
votes
1answer
27 views

Can the attention mechanism improve the performance in the case of short sequences?

I am aware that the attention mechanism can be used to deal with long sequences, where problems related to gradient vanishing and, more generally, representing effectively the whole sequence arise. ...
0
votes
0answers
12 views

Chatbot with Context Management and Awareness

I am currently implementing a closed-domain FAQ chatbot using https://www.sbert.net/index.html as my main model for answering questions from the users. I wish to extend the functionality of the ...
2
votes
0answers
59 views

How to handle long sequences with transformers?

I have a time series sequence with 10 million steps. In step $t$, I have a 400 dimensional feature vector $X_t$ and a scalar value $y_t$ which I want to predict during inference time and I know during ...
2
votes
1answer
55 views

What is different in each head of a multi-head attention mechanism?

I have a difficult time understanding the "multi-head" notion in the original transformer paper. What makes the learning in each head unique? Why doesn't the neural network learn the same ...
1
vote
0answers
4 views

POS Tag Frequency and Language Translation

When we translate a text from one language to another, how does the frequency of various POS tags change? So let's say we have a text in English with 10% nouns, 20% adjectives, 15% adverbs, 25% verbs, ...
1
vote
0answers
59 views

What is the gradient of an attention unit?

The paper Attention Is All You Need describes the Transformer architecture, which describes attention as a function of the queries $Q = x W^Q$, keys $K = x W^K$, and values $V = x W^V$: $\text{...
2
votes
1answer
50 views

What is the cost function of a transformer?

The paper Attention Is All You Need describes the transformer architecture that has an encoder and a decoder. However, I wasn't clear on what the cost function to minimize is for such an architecture. ...
0
votes
0answers
13 views

Finding or creating a dataset for Neural Text Simplification

I'm currently starting a research project focused on NLP. One of the steps involved in this project will be the development of a text simplification system, probably using a neural encoder-decoder ...
0
votes
0answers
21 views

Is it possible to automatically remove ignore (or remove) the equations (and other noisy elements) while performing OCR?

I have academic pdf data. I am using OCR for converting it into text format. The pdf has a few mathematical equations and terms which are acting as noise for my task. Any way through which the task of ...
2
votes
1answer
34 views

Transformers: how to get the output (keys and values) of the encoder?

I was reading the paper Attention Is All You Need. It seems like the last step of the encoder is a LayerNorm(relu(WX + B) + X), i.e. an add + normalization. This should result in a $n$ x $d^{model}$ ...
2
votes
1answer
30 views

Transformers: how does the decoder final layer output the desired token?

In the paper Attention Is All You Need, this section confuses me: In our model, we share the same weight matrix between the two embedding layers [in the encoding section] and the pre-softmax linear ...
2
votes
0answers
36 views

Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?

When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN. Lets say instead of using dense ...
0
votes
1answer
54 views

Making generated texts from “data-to-text” more variable

I am diving in data-to-text generation for long articles (> 1000 words). After creating a template and fill it with data I am currently going down on paragraph level and adding different paragraphs,...
2
votes
1answer
32 views

Predict next event based on previous events and discrete reward values

Suppose, I have several sequences that include a series of text (the length of sequence can be varied). Also, I have some related reward value. however, the value is not continuous like the text. It ...
0
votes
0answers
12 views

What Deep Learning Applications Might Require Super-Computers or “SuperPODs”

With the release of NVIDIA's DGX SuperPOD of A100 GPUs, supercomputers will/are becoming more and more common-place. What potential deep learning tasks/applications might become more accessible with ...
0
votes
0answers
9 views

Best strategy for Classification of Science Subjects. Phy, Chem , Maths and Bio? BERT, Transformers, Attention+SLTM, Self-Attention+LSTM?

I am working on a project where I have to first classify the Subjects of the given question and then the respective Chapter and then the sub-topic. In a nutshell, I have to predict the Subject, Grade ...
0
votes
0answers
10 views

How to make specific test data prediction with fitted GaussianNB Classifier in Python

I'm trying to make news classification. Here is the neural network: ...
1
vote
1answer
283 views

How is Google Translate able to convert texts of different lengths?

According to my experience with Tensorflow and many other frameworks, neural networks have to have a fixed shape for any output, but how does Google translate convert texts of different lengths?

1
2 3 4 5
10