Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

176 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2
votes
0answers
146 views

How do the sine and cosine functions encode position in the transformer?

After going through both the "Illustrated Transformer" and "Annotated Transformer" blog posts, I still don't understand how the sinusoidal encodings are representing the position of elements in the ...
2
votes
1answer
154 views

Anomaly Detection in distributed system using generated log file

I am developing an AI tool for anomaly detection in a distributed system.  The system supports an interface that combines several individual logs into a single log file generating approx. 7000 entries/...
2
votes
0answers
42 views

Searching for dataset in specifix context for NLP, sentiment analysis

I would like to create an NPC engine for games that are in a fantasy context like Lord of the Rings, Warcraft, Skyrim, Dragon Age, etc. This engine should be able to determine the polarity of the ...
2
votes
0answers
38 views

Preset Intent list

I'm trying to build a chat analysis that could identify the intent of check price. Has any kind of preset intent list to train my chatbot already been done ? Data: A data set for "check price" ...
2
votes
0answers
69 views

Sentence classification and named identity detection with automatic retraining

I am learning AI and trying out my first real-life AI application. What I am trying to do is taking as an input various sentences, and then classifying the sentences into one of X number of categories ...
2
votes
0answers
90 views

NLP proved against US legal texts

I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts. Obviously some training would need to be done, but I've found little ...
2
votes
0answers
36 views

What is the “question” when using Dynamic Memory Networks to do part-of-speech tagging and sentiment analysis?

In the paper Ask Me Anything: Dynamic Memory Networks for Natural Language Processing the authors described a Dynamic Memory Network in the context of question answering. Then, they also tested the ...
2
votes
0answers
73 views

Advanced NLG - robot journalist

I want to produce a bot in Python that automatically generates short football summaries from Whoscored data. For my first stage I generate the articles with different sentence templates and lots of ...
2
votes
1answer
78 views

How does transformer leverage GPU which trains faster than RNN?

How does transformer leverage GPU which trains faster than RNN? I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer ...
1
vote
1answer
32 views

Is it possible to classify the subject of a conversation?

I would like to classify the subject of a conversation. I could classify each messages of the conversation, but I will loose some imformation because of related messages. I also need to do it ...
1
vote
1answer
30 views

Can we use NLP to understand/parse/compile programming code?

I wonder if we can use Natural Language Processing (NLP) to process programming code: Given a piece of code, can we Translate it to human language to understand what it does? The input could be a ...
1
vote
0answers
27 views

What are some programming related topics that can be solved using NLP?

I've been working on the Punctuation Restoration Problem for my Master's Thesis, however, me being primarily a programmer at heart, I wish I could use some of my NLP skills to solve issues related to ...
1
vote
1answer
44 views

How do you find the gender of a noun in German using spacy

I am having a difficult time trying to find out if spacy lists the gender or for that matter the inflection of any word in German. The software was started in Berlin so I find it very difficult to ...
1
vote
1answer
51 views

How is BERT different from the original transformer architecture?

As far as I can tell, BERT is a type of Transformer architecture. What I do not understand is: How is Bert different from the original transformer architecture? What tasks are better suited for BERT,...
1
vote
0answers
25 views

How homographs is an NLP task can be treated?

A homograph - is a word that shares the same written form as another word but has a different meaning. They can be even different parts of speech. For example: close(verb) - close(adverb) lead(verb)...
1
vote
0answers
22 views

What are the best techniques to perform text simplification?

I'm evaluating the state of the art techniques to translate legal text to simple text, what are the best approaches for a non-English language (Portuguese)?
1
vote
0answers
58 views

How large should the corpus be to optimally retrain the GPT-2 model?

I just started working with the GPT-2 models and want to retrain one on a pretty narrow topic, so I have problems finding training material. How large should the corpus be to optimally retrain the GPT-...
1
vote
0answers
48 views

Embedding Layer into Convolution Layer

I'm looking to encode PDF documents for deep learning such that an image representation of the PDF refers to word embeddings instead of graphic data So I've indexed a relatively small vocabulary (88 ...
1
vote
1answer
64 views

How is F1 score calculated in question answering system?

I have an NLP model for answer-extraction. So basically I have a paragraph and a question as input and my model extracts the span of the paragraph that corresponds to the answer to the question. I ...
1
vote
0answers
56 views

Is there an optimal way to split the text into small parts when working with co-reference resolution?

I am working with co-reference resolution in a large text. Is there an optimal way to split the text into small parts? Or the best correct procedure is to use the entire text? Just for reference, I am ...
1
vote
1answer
40 views

How can I identify bigrams and trigrams that represent concepts?

I have many text documents and I want to identify concepts in these documents in an unsupervised manner. One of my problems is that the concepts can be bigrams, trigrams, or even longer. So, for ...
1
vote
0answers
24 views

What is the score used to visualize attention in this paper?

I'm reading this paper Global-Locally Self-Attentive Dialogue State Tracker and follow through the implementation published in GLAD. I was wondering if someone can clarify what variable or score is ...
1
vote
1answer
98 views

How to fine tune BERT for question answering?

I wish to train two domain-specific models: Domain 1: Constitution and related Legal Documents Domain 2: Technical and related documents. For Domain 1, I've access to a text-corpus with texts from the ...
1
vote
0answers
15 views

How to autocorelate multiple variants of same text into one?

I want to improve quality of translations for open-source projects in Ukrainian language. We have multiple translations from different authors. We can also translate messages using machine ...
1
vote
1answer
63 views

Can we use a pre trained Encoder (BERT, XLM ) with a Decoder (GPT, Transformer-XL) to build a Chatbot instead of Language Translation?

I was wondering if the BART or T5 models can do the task of generating sentences in English. Most of the models I have mentioned ...
1
vote
0answers
24 views

Low accuracy during training for text summarization

I am trying to implement an extractive text summarization model. I am using keras and tensorflow. I have used bert sentence embeddings and the embeddings are fed into an LSTM layer and then to a Dense ...
1
vote
0answers
27 views

Two questions about the architecture of Google Bert model (in particular about parameters)

I'm looking for someone who can help me clarify a few details regarding the architecture of Bert model. Those details are necessary for me to come with a full understanding of Bert model, so your help ...
1
vote
0answers
33 views

Transformer encoding for regression

I have a string of characters encoding a molecule. I want to regress some properties of those molecules. I tried using an LSTM that encodes all one hot encdoed characters, and then I take the last ...
1
vote
1answer
74 views

Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product attention?

In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix: The reason why is stated that this constrains the ...
1
vote
0answers
19 views

Sentence Segmentation for “Bullets & Numbering”

I am trying to input text into my word processor to be split into sentences first and then into words. An example paragraph: ...
1
vote
0answers
29 views

Which NLP model to use to handle long context?

I'm trying to process product data for an e-commerce platform. The goal is to understand products' size. Just to show you some examples on how messy product dimension description is: ...
1
vote
0answers
33 views

Designing a chatbot personal project with zero coding experience, using an existing platform

My girlfriend has a masters degree in linguistics and would like to create an AI chatbot personal project to show potential employers her linguistics skills since she is struggling to find a job. ...
1
vote
0answers
30 views

Creating Text Features using word2vec

My task is to classify some texts. I have used word2vec to represent text words and I pass them to an LSTM as input. Taking into account that texts do not contain the same number of words, is it a ...
1
vote
0answers
13 views

What are the best datasets available for music information retrieval?

I am interested in doing some work in classification problems in music information retrieval. I know that there are some formats of datasets (such as MIDI, Spectrogram, Piano-roll, MusicXML, etc.) for ...
1
vote
0answers
24 views

Building a spell check model

I have customer review texts. The data consists of the the raw and manually corrected texts of the reviews. I have aligned these pairs by using similarity algorithms and matched the words on them. ...
1
vote
0answers
101 views

GPT-2: (Hardware) requirements for fine-tuning the 774M model

I wonder if there's anyone who has actually succeeded in fine-tuning GPT-2's 774M model without using cloud TPU's. My GeForce RTX 2070 SUPER couldn't handle it in previous attempts. I'm running ...
1
vote
0answers
23 views

How well can NLP techniques recognize connotations in natural languages?

What is the state of the art with respect to recognizing connotations in natural languages? For instance: Trump is a better president than Obama. [Praising] Trump is the worst president ...
1
vote
0answers
16 views

What challenges are there in cross-lingual entity linking?

I am reading the paper entitled Improving Candidate Generation for Low resource Cross-Lingual Entity Linking. Cross-lingual entity linking (XEL) is the task of finding referents in a target language ...
1
vote
0answers
19 views

How do LSTM and GRU avoid to overcome the vanishing gradient problem?

I'm watching the video Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorflow Tutorial | Edureka where the author says that the LSTM and GRU architecture help to reduce the ...
1
vote
0answers
33 views

What are examples of tutorials and blogs for beginners to master the cross-lingual information retrieval?

Currently, I am following the Dan Jurofsky NLP Tutorial and CS 224 Stanford 2019. Can you list tutorials and blogs for beginners to master the cross-lingual information retrieval?
1
vote
0answers
17 views

What are the current research trends in recognizing narrative similarity?

I am currently working on a term paper on the topic of Narrative Similarity, based on Loizos Michael's work "Similarity of Narratives". I am trying to find the latest trends within this field of study ...
1
vote
0answers
19 views

Building a template based NLG system to generate a report from data

I am a newbie to NLP and NLG. I am tasked to develop a system to generate a report based on a given data table. The structure of the report and the flow is predefined. I have researched on several ...
1
vote
0answers
41 views

What are the advantages and disadvantages of extrinsic and perplexity model evaluation in NLP?

In the video Evaluation and Perplexity by Dan Jurafsky, the author talks about extrinsic and perplexity evaluation in the context of natural language processing (NLP). What are the advantages and ...
1
vote
0answers
25 views

What algorithm to use for finding artists/bands in text and differentiating between artists that share the same name

Here's the data I have: Text from articles from various music blogs & music news sites (title, summary, full content, and sometimes tags). I used a couple different NLP/NER tools (nltk, ...
1
vote
0answers
48 views

Use cases for AI inside the software company

This question is a bit philosophic and is about making new use cases for software companies. Let me describe what exist for now, why it is not enough, and what is needed. I know that there are a lot ...
1
vote
0answers
35 views

Is it possible to do token classification using a model such as GPT-2?

I am trying to use PyTorch's transformers as a part of a research project to do sentiment analysis of several types of review data (laptop and restaurant). To do this, my team is taking a token-...
1
vote
1answer
44 views

Why is the sample size of stochastic gradient descent a power of 2?

I watched in the video lecture of cs224: Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 2 – Word Vectors and Word Senses. They take the sample size of the window to be $2^5 = 32$ or $...
1
vote
1answer
77 views

Pros and Cons of Seq2Seq vs Bidirectional RNN

It seems to me that Seq2Seq models and Bidirectional RNNs try to do the same thing. Is that true? Also, when would you recommend one setup over another? Thanks!
1
vote
0answers
32 views

Simple sequential model with LSTM which doesn't converge

I'm actually trying to create a sequential neural network in order to translate a "human" sentence in a "machine" sentence understandable by an algorithm. Like It didn't work, I've try to create a NN ...
1
vote
1answer
33 views

How to produce documents like factset blackline?

Factset blackline reports essentially can compare two 10-Q SEC filings and show you the difference between the two documents. It highlights added items in green and removed items in red + ...