Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
0
votes
1answer
15 views

How can Transformer Networks handle arbitrary length input

Transformer Networks, introduced in this paper is a popular new NLP architecture that is commonly viewed as an alternative to Recurrent Architectures like LSTMs and GRUs. However, having gone through ...
0
votes
1answer
16 views

Get Neural Network to predict a tag/class on a certain word using the surrounding words as context [PyTorch]?

I am somewhat a novice at the topic of Neural Netoworks and PyTorch. I am trying to create a model that takes a word (that I have modified very slightly) and a 'window' of context around it and ...
0
votes
0answers
8 views

Automated Scoring (non-english language) Using BERT

i'm a student and i'm new to NLP. I want to build an Automated Scoring system which is in Indonesian Language using BERT. The system is expected to be able to measure the similarity of an answer(e.g: ...
3
votes
1answer
61 views

How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
0
votes
0answers
22 views

What is the reason for implementing position-wise Feed Forward Network in Transformer?

Does anyone know the reason why there's a FCNN after the self-attention layer? Or at least some intuition for it?
0
votes
1answer
24 views

NLP: What is expected from the output of a perfect coreference system?

For instance, consider the following piece of text: 'The father of Richard is a very nice guy. He was born in a poor family. Because of that, Richard learnt very good values. Richard is also a very ...
-1
votes
0answers
55 views

Is GPT-3 better than what DeepMind could do? [closed]

So there is this tremendous hype about GPT-3, which is in fact impressive. But I've been always convinced that DeepMind was by far the leader in machine learning. However, I didn't find any comparable ...
2
votes
1answer
51 views

How to predict the “word” based on the meaning in a document?

What I mean to say is For example, if I give the meaning of Apple from the dictionary as input to the program, it should give output as Apple. Or I say My day to day job involves monitoring and ...
0
votes
1answer
67 views

How to train an LSTM with varying length input?

I have a dataset where each of the training instances is different in the length and the data is sequential. So, I design an LSTM but I am thinking about how to train the LSTM. In fixed-length data, ...
0
votes
0answers
43 views

Using transformer but masking in reverse direction/smart sampling for desired final word?

I'm trying to generate rhymes, so it would be very helpful to have a language model where I could input a final word, and have it output a sequence of words that ends with that word. I could train my ...
1
vote
0answers
53 views

How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
1
vote
0answers
45 views

Embedding Layer into Convolution Layer

I'm looking to encode PDF documents for deep learning such that an image representation of the PDF refers to word embeddings instead of graphic data So I've indexed a relatively small vocabulary (88 ...
1
vote
1answer
52 views

How is F1 score calculated in question answering system?

I have an NLP model for answer-extraction. So basically I have a paragraph and a question as input and my model extracts the span of the paragraph that corresponds to the answer to the question. I ...
1
vote
0answers
56 views

Is there an optimal way to split the text into small parts when working with co-reference resolution?

I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text? Just for reference, I am ...
0
votes
0answers
31 views

What could be the possible strategy and Deep Learning method that MathPix might be using for LaTex detection?

I want to build an open Source OCR just like MathPix. There is already a model to extract LaTex from the image by Harverd NLP's im2markup but the problem is that their data has been trained and tested ...
0
votes
0answers
32 views

Suggestion for an Enterprise search engine: What are all the way I can use machine learning/dnn/nlp to enhance elasticsearch results?

We are building an enterprise search engine for about 130,000 documents. It includes FAQs, Small articles to 3-4 page documents, all of them are stored in structured JSON. We were originally planning ...
0
votes
0answers
26 views

Do we have to use the IOB format on labels in the NER dataset? If so, why?

Do we have to use the IOB format on labels in the NER dataset (such as B-PERSON, I-PERSON, etc.) instead of using the usual format (PERSON, ORGANIZATION, etc.)? If so, why? How will it affect the ...
1
vote
1answer
37 views

How can I identify bigrams and trigrams that represent concepts?

I have many text documents and I want to identify concepts in these documents in an unsupervised manner. One of my problems is that the concepts can be bigrams, trigrams, or even longer. So, for ...
1
vote
0answers
24 views

What is the score used to visualize attention in this paper?

I'm reading this paper Global-Locally Self-Attentive Dialogue State Tracker and follow through the implementation published in GLAD. I was wondering if someone can clarify what variable or score is ...
0
votes
0answers
21 views

Comparing English and Spanish Queries using NLP

I am trying to see whether there are differences in sentences between the topics in 2 languages ie English and Spanish. Eg. (Face masks are mandatory, la mascarilla es obligatoria) but this stretched ...
1
vote
1answer
71 views

How to fine tune BERT for question answering?

I wish to train two domain-specific models: Domain 1: Constitution and related Legal Documents Domain 2: Technical and related documents. For Domain 1, I've access to a text-corpus with texts from the ...
0
votes
1answer
24 views

Text classification of non-equal length texts, should I pad left or right?

Text classification of equal length texts works without padding, but in reality, practically, texts never have the same length. For example, spam filtering on blog article: ...
0
votes
0answers
21 views

Looking for an Approach method for News maker

I have some databases with information about exports and imports of a country, what are they importing, the times that an importation was made, the country from and target, everything related to ...
0
votes
1answer
22 views

Is it possible to create a named entity recognition system without using POS tagging in the corpus?

Is it possible to create a named entity recognition system without using POS tagging in the corpus?
0
votes
1answer
29 views

NLP Identifying important key words in a corpus

I am intrigued with the idea of Zettelkasten but unsatisfied with the current implementations. It seems to me that a machine learning and NLP approach could be productive by helpfully identifying “...
2
votes
1answer
35 views

Do transformers have success in other domains different than NLP?

Everybody knows how successful transformers have been in NLP. Is there known work on other domains (e.g that also have a sequential natural way of occurring, such as stock price prediction or other ...
0
votes
1answer
24 views

What are the main differences between a language model and a machine translation model?

What are the main differences between a language model and a machine translation model?
0
votes
1answer
34 views

How to make spacy lemmatization process fast?

I am applying spacy lemmatization on my dataset, but already 20-30 mins passed and the code is still running. Is there anyway to make it faster? Is there any option to do this process using GPU? My ...
1
vote
1answer
42 views

How to use speaker's information as well as text for fine-tuning BERT?

I want to classify my corporate chat messages into a few categories such as question, answer, and report. I used a fine-tuned BERT model, and the result wasn't bad. Now, I started thinking about ways ...
2
votes
2answers
210 views

What are the main ideas behind NER?

Named entity recognition (NER), also known as entity chunking/extraction, is a popular technique used in information extraction to identify and segment the named entities and classify or categorize ...
1
vote
0answers
15 views

How to autocorelate multiple variants of same text into one?

I want to improve quality of translations for open-source projects in Ukrainian language. We have multiple translations from different authors. We can also translate messages using machine ...
0
votes
0answers
25 views

How to implement NLP for ecommerce product search

I'm implementing an ecommerce search using nlp. there is a lack of clear documentation anywhere online about implementation and flow. this is what i'm doing. Search query comes in Do spellcheck Do ...
1
vote
1answer
55 views

Can we use a pre trained Encoder (BERT, XLM ) with a Decoder (GPT, Transformer-XL) to build a Chatbot instead of Language Translation?

I was wondering if the BART or T5 models can do the task of generating sentences in English. Most of the models I have mentioned ...
1
vote
1answer
58 views

When to use NLP, NLG and NLU in conversation agents?

I had read some blogs (like 1, 2 or 3) about what the difference between all three of them is. I am trying to build an open domain conversation agent using natural language AI. That agent can do ...
0
votes
0answers
21 views

How to find difference between topics in two languages using NLP

I want to analyze queries and their differences between two different languages English and Spanish in this case. I'm aware of topic modeling. I'm in search of any corpus available or any algorithm to ...
0
votes
0answers
32 views

Which NLP Parser for Morse conversations?

In ham radio, Morse code is used in CW transmissions (yes even today). My project is to take already decoded Morse code text strings from an existing decoder and then recognize different ...
3
votes
1answer
82 views

What are the keys and values of the attention model for the encoder and decoder in the “Attention Is All You Need” paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
0
votes
1answer
27 views

Why does the BERT NSP head linear layer have two outputs?

Here's the code in question. https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L491 ...
1
vote
0answers
24 views

Low accuracy during training for text summarization

I am trying to implement an extractive text summarization model. I am using keras and tensorflow. I have used bert sentence embeddings and the embeddings are fed into an LSTM layer and then to a Dense ...
1
vote
0answers
26 views

Two questions about the architecture of Google Bert model (in particular about parameters)

I'm looking for someone who can help me clarify a few details regarding the architecture of Bert model. Those details are necessary for me to come with a full understanding of Bert model, so your help ...
3
votes
1answer
83 views

What is the intuition behind the attention mechanism?

Attention idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to "look back” at the complete input and extracts ...
1
vote
0answers
32 views

Transformer encoding for regression

I have a string of characters encoding a molecule. I want to regress some properties of those molecules. I tried using an LSTM that encodes all one hot encdoed characters, and then I take the last ...
1
vote
1answer
62 views

Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product attention?

In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix: The reason why is stated that this constrains the ...
3
votes
1answer
41 views

How can I make ELIZA more realistic?

I’ve coded a simple ELIZA chatbot for a high school coding competition. The chatbot is part of an app that’s designed to help its user cope with depression, anxiety, and similar mental health ...
4
votes
1answer
69 views

How is the Jacobian a generalisation of the gradient?

I came across these slides Natural Language Processing with Deep Learning CS224N/Ling284, in the context of natural language processing, which talk about the Jacobian as a generalization of the ...
1
vote
0answers
17 views

Sentence Segmentation for “Bullets & Numbering”

I am trying to input text into my word processor to be split into sentences first and then into words. An example paragraph: ...
0
votes
0answers
21 views

What is the right way to set the dimension of the word representation in SkipGram word2vec?

I know that in word2vec each word has two word representations; one for the center word and one for the context word. What is the right way to set the dimension of the center word using SkipGram, ...
1
vote
0answers
29 views

Which NLP model to use to handle long context?

I'm trying to process product data for an e-commerce platform. The goal is to understand products' size. Just to show you some examples on how messy product dimension description is: ...
1
vote
0answers
32 views

Designing a chatbot personal project with zero coding experience, using an existing platform

My girlfriend has a masters degree in linguistics and would like to create an AI chatbot personal project to show potential employers her linguistics skills since she is struggling to find a job. ...
1
vote
0answers
30 views

Creating Text Features using word2vec

My task is to classify some texts. I have used word2vec to represent text words and I pass them to an LSTM as input. Taking into account that texts do not contain the same number of words, is it a ...

1
2 3 4 5
9