Questions tagged [natural-language-processing]

For questions related to natural language processing (NLP), which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

176 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
0answers
24 views

Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?

BERT encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of ...
4
votes
1answer
568 views

How to use BERT as a multi-purpose conversational AI?

I'm looking to make an NLP model that can achieve a dual purpose. One purpose is that it can hold interesting conversations (conversational AI), and another being that it can do intent classification ...
4
votes
0answers
49 views

What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?

I was reading the paper by Kalchbrenner et al. titled A Convolutional Neural Network for Modelling Sentences and I am struggling to understand their definition of convolutional layer. First, let's ...
4
votes
1answer
110 views

Is it possible to verify SOP of call center conversation using AI?

As we know in call centers there are certain SOP(standard operating procedure), for example few are below, Agent greeted customer Agent verified the validity of customer before providing sensitive ...
3
votes
1answer
89 views

What are the keys and values of the attention model for the encoder and decoder in the “Attention Is All You Need” paper?

I have recently encountered the paper on NLP. It is very new to me and I am still unable to see how that works. I have used all the resources over there from the original paper to Youtube videos and ...
3
votes
1answer
33 views

What is meant by the expected BLEU cost when training with BLEU and SIMILE?

Recently I was reading a paper based on a new evaluation metric SIMILE. In a section, validation loss comparison had been made for SIMILE and BLEU. The plot showed the expected BLEU cost when training ...
3
votes
0answers
33 views

Could zero-padding affect learning in a negative way?

I implemented an LSTM with Keras to perform word ordering task (given a syntactically unordered sentence, the goal is to label ...
3
votes
0answers
32 views

Creating an AI than can learn to give instructions

So we think a computer is dumb because it can only follow instructions. Therefor I am trying to create an AI that can give instructions. The idea is this: Create a geometric scene (A) then make a ...
3
votes
0answers
28 views

Why embedding layer is used in the character-level Natural Language Processing models

Problem Background I am working with a problem, which requires a character-level, deep learning model. Previously I was working with word-level deep NLP (Natural Language Processing) models, and in ...
3
votes
0answers
23 views

Can computers recognise “grouping” from voice tonality?

In human communication, tonality or tonal language play many complex information, including emotions and motives. But excluding such complex aspects, tonality serves some a very basic purpose of "...
3
votes
0answers
141 views

How does bidirectional encoding allow the predicted word to indirectly “see itself”?

Before the release of BERT, we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that's ...
3
votes
0answers
24 views

Why does all of NLP literature use noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
3
votes
0answers
34 views

How would an AI visualize a story written in natural language?

Can AI transform natural language text describing real scenarios to visual images and videos ? How does as AI interprets say a Harry Potter story if it has to reproduce it in form of videos ? Would be ...
3
votes
0answers
38 views

Extracting referenced documents

I'm looking to write an AI that will be able to extract in text references from standards documents to assist human research. My use case is extracting the identifying numbers, for example, "AR 25-2",...
3
votes
0answers
158 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in The Unreasonable Effectiveness of Recurrent Neural Networks) that when training RNN on text for something like language modeling, the text is usually featurized character-by-...
3
votes
0answers
391 views

RL to generate sentences

I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. Eg: Input: capital, Paris, ...
3
votes
0answers
207 views

Has anyone used YodaQA for natural language processing?

Has anyone used YodaQA for natural language processing? How easy is it to link to a document database other than Wikipedia? We're thinking we can create a bot to use AI to analyze our developer and ...
2
votes
0answers
26 views

Estimating an $n$-Gram model using on bigrams

One of the main arguments against $n$-gram models is that, as $n$ increases, there is no way to compute $P(w_n|w_1,\cdots,w_{n-1})$ from training data (since the change of visiting $w_n,...,w_1$ is ...
2
votes
0answers
39 views

NLP Bible verse division problem: Whats the best model/method?

I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
2
votes
0answers
56 views

Is there a good book or paper on word embeddings?

Is there a good and modern book that focuses on word embeddings and their applications? It would also be ok to provide the name of a paper that provides a good overview of word embeddings.
2
votes
0answers
29 views

How should I design a reward function for a NLP problem where two models interoperate?

I would like to design a reward function. I am training two models from the first model that classify set of texts (paragraphs and keywords) and I also got some hidden states. The second model is ...
2
votes
0answers
26 views

How does one detect linguistic recursion so as to know how much nesting there is, if any?

To be clear, recursion in linguistics is here better called "nesting" in this CS context to avoid confusing it with the other recursion. How does one detect nesting? I am particularly ...
2
votes
0answers
46 views

Pre-trained Models for Topic Modelling Transfer Learning (LDA)

I've been searching online - and so far, I've been unable to find any publicly-accessible pre-trained models that can be used for LDA Topic Modeling - Transfer Learning. Can anyone share any resources ...
2
votes
0answers
31 views

How does positional encoding work in the transformer model?

In the transformer model, to incorporate positional information of texts, the researchers have added a positional encoding to the model. How does positional encoding work? How does the positional ...
2
votes
0answers
19 views

Sign Language to Speech conversion

Is there any solution about sign language to speech conversion for mobiles? Can anyone suggest me the flow and tools so that I may implement the solution for mobiles?
2
votes
0answers
52 views

Can we use GPT-2 to smooth out / correct text?

Are we able to use models like GPT-2 to smooth out/correct text? For instance if I have two paragraphs that need some text to make the transition easier to read, could this text be generated? And, ...
2
votes
0answers
22 views

How to tell if two hotel reviews addressing the same thing

I am playing with a large dataset of hotel reviews, which contains both positive and negative reviews (the reviews are labeled). I want to use this dataset to perform textual style transfer - given a ...
2
votes
0answers
128 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
2
votes
0answers
36 views

How can I extract the reason of the legal compensation from a court report?

I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report) Order mister X ...
2
votes
0answers
83 views

Is NLP likely to be sufficiently solved in the next few years?

The reason I am asking this question is because I am about to start a PhD in NLP. So I am wondering if there would be as much job opportunities in research in industry as oppose to in academia in the ...
2
votes
0answers
39 views

What role do distractors play in natural language processing?

I’m doing research on natural language processing (NLP). I’d like to put together my own model. However, I'm running into a concept I am not familiar with, namely, distractors. A google search does ...
2
votes
0answers
38 views

How can I feed any word into a neural network?

I am working on an Intent detection problem for a chatbot in Java. So I need to convert words from String to a double[] format. I tried using wordToVec(deeplearning4j), but it does not return a vector ...
2
votes
0answers
18 views

NLP annotation tool online and other tools to compare performances of different NLP algorithms

I do text annotations (POS tagging, NER, chunking, synset) by using a specific annotation tool for Natural Language Processing. I would like to make the same annotations on different tools to compare ...
2
votes
0answers
30 views

Is there a detailed description or implementation of an end-to-end speech recognition system?

I am currently trying to implement an end-to-end speech recognition system from scratch, that is, without using any of the existing frameworks (like TensorFlow, Keras, etc.). I am building my own ...
2
votes
0answers
34 views

What is the underlying model of IBM Watson Assistant and Microsoft LUIS?

As I stated in my question, I would like to know the underlying pipeline and machine learning models that are used to classify intents and identify entities in IBM Watson Assistant and Microsoft LUIS ...
2
votes
0answers
24 views

How to train a transformer text-to-text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
2
votes
0answers
31 views

What is the State-of-the-Art open source Voice Cloning tool right now?

I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning),...
2
votes
0answers
17 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
0answers
46 views

Grouped Text classification

I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph ...
2
votes
0answers
56 views

Machine learning methods to identify the recipient of a document?

I need some advice on what AI methods would be suited to the identification of a recipient of a document, where the format of the documents may vary.
2
votes
0answers
124 views

What is a conditional random field?

I new in machine learning, especially in Conditional Random Fields (CRF). I have read several articles and papers and in there is always associated with HMM and sequences classification. I don't ...
2
votes
0answers
41 views

Models to extract Causal Relationship between entities in a document using Natural Language Processing techniques

I am looking to extract causal relations between entities like Drug and Adverse Effect in a document. Are there any proven NLP or AI techniques to handle the same. Also are there ways to handle cases ...
2
votes
0answers
80 views

Bert super easy implementation

I myself am not new to NLP, but for some reason I am unable to grasp purity of BERT. I have seen a ton of blogs, github repos, but none could clarify BERT usage to me. It would be helpful if you ...
2
votes
0answers
56 views

ChatBot applications [Academia]

I am in education and I'm wondering about the use of chatbot-like tools to facilitate automated discussions among students. The chatbot domain of knowledge would be purposely restricted to a specific ...
2
votes
0answers
39 views

How can I generate a document from a single word using GPT or BERT?

I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it. I came across language models GPT,...
2
votes
0answers
31 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
2
votes
0answers
151 views

Is Reinforcement Learning the future of Natural Language Processing?

I was reading about the grounding problem after seeing it mentioned in another answer today. The article states that, in order to avoid the "infinite regress" of defining all words with other words, ...
2
votes
0answers
152 views

Why does the BERT encoder have an intermediate layer between the attention and neural network layers with a bigger output?

I am reading the BERT paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. As I look at the attention mechanism, I don't understand why in the BERT encoder we have ...
2
votes
0answers
88 views

How to perform unsupervised anomaly detection from log file with mostly textual data?

I have a log file of the format, Index, Date, Timestamp, Module, App, Context, Session, Verbosity level, Description The log file can be considered as a master log, which consists of individual ...
2
votes
0answers
30 views

Translate product names with AI

I frequently need to translate product names for hundreds of similar products -- and I have a list of past product names. Is it possible to train AI to review past translations and translate? It doesn'...