Questions tagged [natural-language-processing]

For questions related to natural language processing, which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

118 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
11
votes
3answers
956 views

What topologies are largely unexplored in machine learning?

Geometry and AI Matrices, cubes, layers, stacks, and hierarchies are what we could accurately call topologies. Consider topology in this context the higher level geometrical design of a learning ...
7
votes
1answer
527 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
6
votes
1answer
71 views

What are the current big challenges in Natural Language Processing and Understanding?

I'm doing a paper for a class on the topic of big problems that are still prevalent in AI, specifically in the area of Natural Language Processing and Understanding. From what I understand, the areas: ...
4
votes
0answers
61 views

Are embeddings in multi-lingual language models comparable across languages?

Facebook has just pushed out a bigger version of their multi-lingual language model XLM, called XLM-R. My question is: do these kind of multi-lingual models imply, or even ensure, that their ...
4
votes
1answer
64 views

Will BERT embedding be always same for a given document when used as a feature extractor

When we use BERT embeddings for a classification task, would we get different embeddings every time we pass the same text through the BERT architecture? If yes, is it the right way to use the ...
4
votes
0answers
44 views

What is the motivation for row-wise convolution and folding in Kalchbrenner et al. (2014)?

I was reading the paper by Kalchbrenner et al. titled A Convolutional Neural Network for Modelling Sentences and I am struggling to understand their definition of convolutional layer. First, let's ...
3
votes
0answers
17 views

Why embedding layer is used in the character-level Natural Language Processing models

Problem Background I am working with a problem, which requires a character-level, deep learning model. Previously I was working with word-level deep NLP (Natural Language Processing) models, and in ...
3
votes
0answers
22 views

Can computers recognise “grouping” from voice tonality?

In human communication, tonality or tonal language play many complex information, including emotions and motives. But excluding such complex aspects, tonality serves some a very basic purpose of "...
3
votes
0answers
20 views

Why does all of NLP literature use noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
3
votes
0answers
31 views

How would an AI visualize a story written in natural language?

Can AI transform natural language text describing real scenarios to visual images and videos ? How does as AI interprets say a Harry Potter story if it has to reproduce it in form of videos ? Would be ...
3
votes
2answers
251 views

Handling emotion in informal text (Hi vs HIIIIII!!!!)?

This is a question related to Neural network to detect "spam"?. I'm wondering how it would be possible to handle the emotion conveyed in text. In informal writing, especially among a ...
3
votes
1answer
94 views

Is it possible to verify SOP of call center conversation using AI?

As we know in call centers there are certain SOP(standard operating procedure), for example few are below, Agent greeted customer Agent verified the validity of customer before providing sensitive ...
3
votes
0answers
122 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
3
votes
0answers
372 views

RL to generate sentences

I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. Eg: Input: capital, Paris, ...
2
votes
0answers
29 views

How can I feed any word into a neural network?

I am working on an Intent detection problem for a chatbot in Java. So I need to convert words from String to a double[] format. I tried using wordToVec(deeplearning4j), but it does not return a vector ...
2
votes
0answers
10 views

NLP annotation tool online and other tools to compare performances of different NLP algorithms

I do text annotations (POS tagging, NER, chunking, synset) by using a specific annotation tool for Natural Language Processing. I would like to make the same annotations on different tools to compare ...
2
votes
0answers
28 views

Is there a detailed description or implementation of an end-to-end speech recognition system?

I am currently trying to implement an end-to-end speech recognition system from scratch, that is, without using any of the existing frameworks (like TensorFlow, Keras, etc.). I am building my own ...
2
votes
0answers
23 views

Is there a tool to convert from the brat standoff format to CoNLL-U format?

I've been searching for a tool to convert from the brat standoff format to the CoNLL-U format, so that to use it as parsing corpus model to the spaCy library. Can you help me?
2
votes
0answers
30 views

What is the underlying model of IBM Watson Assistant and Microsoft LUIS?

As I stated in my question, I would like to know the underlying pipeline and machine learning models that are used to classify intents and identify entities in IBM Watson Assistant and Microsoft LUIS ...
2
votes
0answers
22 views

How to train a transformer text-to-text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
2
votes
1answer
36 views

What is the best way to find the similarities between two text documents?

I would like to develop a platform in which people will write text and upload images. I am going to use Google API to classify the text and extract from the image all kinds of metadata. In the end, I ...
2
votes
0answers
25 views

What is the State-of-the-Art open source Voice Cloning tool right now?

I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning),...
2
votes
0answers
16 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
0answers
40 views

Grouped Text classification

I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph ...
2
votes
0answers
53 views

Machine learning methods to identify the recipient of a document?

I need some advice on what AI methods would be suited to the identification of a recipient of a document, where the format of the documents may vary.
2
votes
0answers
68 views

What is a conditional random field?

I new in machine learning, especially in Conditional Random Fields (CRF). I have read several articles and papers and in there is always associated with HMM and sequences classification. I don't ...
2
votes
0answers
28 views

Models to extract Causal Relationship between entities in a document using Natural Language Processing techniques

I am looking to extract causal relations between entities like Drug and Adverse Effect in a document. Are there any proven NLP or AI techniques to handle the same. Also are there ways to handle cases ...
2
votes
0answers
59 views

Is there a simple and understandable implementation of BERT?

I am not new to NLP, but, for some reason, I am unable to use BERT. I have seen a ton of blogs, Github repositories, but none could clarify BERT usage to me. It would be helpful if you could provide ...
2
votes
0answers
51 views

ChatBot applications [Academia]

I am in education and I'm wondering about the use of chatbot-like tools to facilitate automated discussions among students. The chatbot domain of knowledge would be purposely restricted to a specific ...
2
votes
0answers
89 views

How does bidirectional encoding allow the predicted word to indirectly “see itself”?

Before the release of BERT, we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that's ...
2
votes
0answers
34 views

How can I generate a document from a single word using GPT or BERT?

I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it. I came across language models GPT,...
2
votes
0answers
18 views

Multi-field text input for LSTM

I'm using LSTM to categorize medium-sized pieces of text. Each item to be categorized has several free-form text fields, in addition to several categorical fields. What is the best approach to using ...
2
votes
2answers
330 views

How can I make meaningful English sentences from given set of words?

I have a set of topics and each topic consists of a set of words. I want to make meaningful English sentences from these words. Each topic consist of 5 to 10 words and these words are relevant to each ...
2
votes
1answer
84 views

How do I use neural networks to implement a chatbot?

I don't know anything about neural networks, but I got the information that making a chatbot with a neural network is very good. Is this really true? What do I need to know in order to build a chat ...
2
votes
0answers
91 views

Why does the BERT encoder have an intermediate layer between the attention and neural network layers with a bigger output?

I am reading the BERT paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. As I look at the attention mechanism, I don't understand why in the BERT encoder we have ...
2
votes
0answers
42 views

How to perform unsupervised anomaly detection from log file with mostly textual data?

I have a log file of the format, Index, Date, Timestamp, Module, App, Context, Session, Verbosity level, Description The log file can be considered as a master log, which consists of individual ...
2
votes
0answers
27 views

Translate product names with AI

I frequently need to translate product names for hundreds of similar products -- and I have a list of past product names. Is it possible to train AI to review past translations and translate? It doesn'...
2
votes
0answers
96 views

How do the sine and cosine functions encode position in the transformer?

After going through both the "Illustrated Transformer" and "Annotated Transformer" blog posts, I still don't understand how the sinusoidal encodings are representing the position of elements in the ...
2
votes
1answer
101 views

Anomaly Detection in distributed system using generated log file

I am developing an AI tool for anomaly detection in a distributed system.  The system supports an interface that combines several individual logs into a single log file generating approx. 7000 entries/...
2
votes
0answers
36 views

Extracting referenced documents

I'm looking to write an AI that will be able to extract in text references from standards documents to assist human research. My use case is extracting the identifying numbers, for example, "AR 25-2",...
2
votes
0answers
82 views

NLP proved against US legal texts

I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts. Obviously some training would need to be done, but I've found little ...
2
votes
0answers
35 views

What is the “question” when using Dynamic Memory Networks to do part-of-speech tagging and sentiment analysis?

In the paper Ask Me Anything: Dynamic Memory Networks for Natural Language Processing the authors described a Dynamic Memory Network in the context of question answering. Then, they also tested the ...
2
votes
0answers
70 views

Advanced NLG - robot journalist

I want to produce a bot in Python that automatically generates short football summaries from Whoscored data. For my first stage I generate the articles with different sentence templates and lots of ...
2
votes
1answer
191 views

Using YodaQA with a non-Wikipedia source

Has anyone used YodaQA for natural language processing? How easy is it to link to a document database other than Wikipedia? We're thinking we can create a bot to use AI to analyze our developer and ...
1
vote
0answers
11 views

Deep audio fingerprinting for word search

Simply speaking, I'm trying to somehow search an audio clip for a list of words, and if found, I mark the time stamps. My use-case is profanity check with a list of pre-defined profane words. Is ...
1
vote
0answers
13 views

How to use TPU for real-time low-latency inference?

I use Google's Cloud TPU hardware extensively using Tensorflow for training models and inference, however, when I run inference I do it in large batches. The TPU takes about 3 minutes to warm up ...
1
vote
1answer
90 views

How to Extract Information from the Image

I'm trying to extract some particular information from the image(png). I tried to extract the text using the below code ...
1
vote
0answers
27 views

Finetuning GPT-2 twice for particular style of writing on a particular topic

Sorry if this is a stupid question. I'm just starting out in ML and am working with gpt-2 for text generation. My situation is that I have to generate text in a particular field for eg. family ...
1
vote
0answers
19 views

Why hasn't deep learning been used for word level alignment?

I've been exploring word-level alignments tools such as MGIZA and it seems to me that there hasn't been any new tool for this problem. Are neural networks not suitable to solve this problem or simply ...
1
vote
0answers
7 views

How can we use Dependency Parsers for Negation detection

I am building a negation detection system. How to use dependency parsers for the same. I am using SPACY for dependency parser