Questions tagged [natural-language-processing]

For questions related to natural language processing, which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
2
votes
0answers
10 views

Can Bert be used to extract embedding for large categorical features?

I've lot of training data points (i.e in millions) and I've around few features but the issue with that is all the features are categorical data with 1 million+ categories in each. So, I couldn't use ...
1
vote
0answers
28 views

How much knowledge of the world is learnt through words?

We know a lot of common sense about the world. Things like "to buy something you need money". I wonder how much of this common sense comes about through actual someone explicitly telling you the ...
4
votes
0answers
37 views

Is Sanskrit still relevant for NLP/AI?

I came across a news article from 2018 where the president of India was saying that Sanskrit is the best language for ML/AI. I have no idea regarding his qualification on either AI or Sanskrit to say ...
4
votes
0answers
23 views

Training an RNN to answer simple quesitons

I would like to train an RNN to follow the sentences: "Would you like some cheese"? with "Yes, I would like some cheese." So whenever the template "Would you like some ____?" appears then RNN ...
0
votes
1answer
24 views

Extract product information from email receipt HTML

I am trying to extract product information from email receipts HTML. Most services I have found focus on OCR from paper receipts or PDFs. I would imagine that extraction of product information would ...
2
votes
0answers
32 views

How can I extract the reason of the legal compensation from a court report?

I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report) Order mister X ...
4
votes
2answers
56 views

Does summing up word vectors destroy their meaning?

For example, I have a paragraph which I want to classify in a binary manner. But because the inputs have to have a fixed length, I need to ensure that every paragraph is represented by a uniform ...
3
votes
0answers
24 views

Creating an AI than can learn to give instructions

So we think a computer is dumb because it can only follow instructions. Therefor I am trying to create an AI that can give instructions. The idea is this: Create a geometric scene (A) then make a ...
4
votes
2answers
48 views

Is there any classifier that works best in general for NLP based projects?

I've written a program to analyse a given piece of text from a website and make conclusary classifications as to its validity. The code basically vectorizes the description (taken from the HTML of a ...
2
votes
1answer
51 views

How to use AI for language recognition?

Given an audio track, I'm trying to find a way to recognize the audio language. Only within a small set (e.g. English vs Spanish). Is there a simple solution to detect the language in a speech?
2
votes
0answers
71 views

Is NLP likely to be sufficiently solved in the next few years?

The reason I am asking this question is because I am about to start a PhD in NLP. So I am wondering if there would be as much job opportunities in research in industry as oppose to in academia in the ...
2
votes
0answers
35 views

What role do distractors play in natural language processing?

I’m doing research on natural language processing (NLP). I’d like to put together my own model. However, I'm running into a concept I am not familiar with, namely, distractors. A google search does ...
0
votes
0answers
15 views

How would an AI learn the concept of the words “repeat twice”?

In a hypothetical conversation: Person A - "Repeat the word 'cat' twice". Person B - "cat cat". I'm thinking about how a human or AI can learn the concept of "...
2
votes
1answer
51 views

Giving an AI a purpose to talk

I am trying to teach my AI to talk. The problem is I'm struggling to find a good scenario in which it needs to. Some ideas I had were: "Describe a geometric scene" - Then together with a parser we ...
1
vote
0answers
26 views

Are there any approaches other than deep learning to deal with unexpected questions in a question answering system?

I'm working on a question answering bot as my graduation project. The main concept is having a text file with many sentences, and building a question answering bot which answers a user's question ...
4
votes
2answers
71 views

How to use BERT as a Multi-Purpose Conversational AI?

After doing some more research, I thought I'd reframe my question a little. I’m looking to make an NLP model that can achieve a dual purpose. One purpose being that it can hold interesting ...
1
vote
1answer
20 views

Can a character-level Seq2Seq setup learn to perfectly reconstruct structured data like name strings?

If not perfect, how well can they do? For example, if I give the Seq2Seq setup a name it did not see in the training process, can it output the same name without error? Example ...
1
vote
1answer
30 views

How does transformer leverage GPU which trains faster than RNN?

How does transformer leverage GPU which trains faster than RNN? I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer ...
0
votes
1answer
31 views

What is the most accurate pretrained sentiment analysis model by 2019?

I've been using OpenAI's 2017 Sentiment Neuron implementation (https://github.com/openai/generating-reviews-discovering-sentiment) for a while, because it was easy to set up and was the most accurate ...
0
votes
1answer
52 views

Need examples for the following definitions

I am currently reading the paper "Similarity of Narratives" by Loizos Michael (link below) and I am having a hard time figuring out the definitions listed (p.107 - p.109). Could someone please give ...
1
vote
1answer
47 views

How to detect any native language when written in Latin characters?

Assume somebody knows only to write in Latin characters. If they write words of any other language (example: Hindi, French, Latin) using the Latin alphabet, how can I detect that language? Example: ...
0
votes
0answers
22 views

Methods for automated insertion of emojis into text?

I want to build a deep learning algorithm that places emojis in a extended tracts of text. The intent is to choose emojis related to the meaning of the text. Where should I start? I've read that ...
8
votes
1answer
123 views

What are the current big challenges in Natural Language Processing and Understanding?

I'm doing a paper for a class on the topic of big problems that are still prevalent in AI, specifically in the area of Natural Language Processing and Understanding. From what I understand, the areas: ...
2
votes
0answers
34 views

How can I feed any word into a neural network?

I am working on an Intent detection problem for a chatbot in Java. So I need to convert words from String to a double[] format. I tried using wordToVec(deeplearning4j), but it does not return a vector ...
2
votes
0answers
11 views

NLP annotation tool online and other tools to compare performances of different NLP algorithms

I do text annotations (POS tagging, NER, chunking, synset) by using a specific annotation tool for Natural Language Processing. I would like to make the same annotations on different tools to compare ...
3
votes
0answers
21 views

Why embedding layer is used in the character-level Natural Language Processing models

Problem Background I am working with a problem, which requires a character-level, deep learning model. Previously I was working with word-level deep NLP (Natural Language Processing) models, and in ...
1
vote
1answer
30 views

Is the Mask Needed for Masked Self-Attention During Inference with GPT-2

My understanding is that masked self-attention is necessary during training of GPT-2, as otherwise it would be able to directly see the correct next output at each iteration. My question is whether ...
1
vote
0answers
26 views

Is there anywhere to get a free dictionary with definitions? [closed]

An ideal thing to teach an AI about what words mean would be a dictionary (or an encyclopedia). Even better would be a child's dictionary. (i.e. "a cat is a furry animal" rather than "a cat is a type ...
2
votes
0answers
28 views

Is there a detailed description or implementation of an end-to-end speech recognition system?

I am currently trying to implement an end-to-end speech recognition system from scratch, that is, without using any of the existing frameworks (like TensorFlow, Keras, etc.). I am building my own ...
1
vote
1answer
16 views

Deep audio fingerprinting for word search

Simply speaking, I'm trying to somehow search an audio clip for a list of words, and if found, I mark the time stamps. My use-case is profanity check with a list of pre-defined profane words. Is ...
4
votes
0answers
62 views

Are embeddings in multi-lingual language models comparable across languages?

Facebook has just pushed out a bigger version of their multi-lingual language model XLM, called XLM-R. My question is: do these kind of multi-lingual models imply, or even ensure, that their ...
2
votes
1answer
36 views

What happens when the output length in the brevity penalty is zero?

The brevity penalty is defined as $$bp = e^{(1- r/c)},$$ where $r$ is the reference length and $c$ is the output length. But what happens if the output length gets zero? Is there any standard way ...
0
votes
0answers
36 views

How to create an automatic Aimbot coach?

In the gaming community an Aimbot is used to extend the human capabilities of playing a game. An average human user has only a limited capabilities of mastering a game. The amount of actions per ...
2
votes
1answer
26 views

How do you build a language model to predict the contextual similarity between two documents?

How do you build a language model to predict the contextual similarity between two documents?
1
vote
0answers
17 views

How to use TPU for real-time low-latency inference?

I use Google's Cloud TPU hardware extensively using Tensorflow for training models and inference, however, when I run inference I do it in large batches. The TPU takes about 3 minutes to warm up ...
2
votes
1answer
105 views

How to Extract Information from the Image

I'm trying to extract some particular information from the image(png). I tried to extract the text using the below code ...
2
votes
0answers
35 views

Is there a tool to convert from the brat standoff format to CoNLL-U format?

I've been searching for a tool to convert from the brat standoff format to the CoNLL-U format, so that to use it as parsing corpus model to the spaCy library. Can you help me?
2
votes
0answers
30 views

What is the underlying model of IBM Watson Assistant and Microsoft LUIS?

As I stated in my question, I would like to know the underlying pipeline and machine learning models that are used to classify intents and identify entities in IBM Watson Assistant and Microsoft LUIS ...
1
vote
1answer
64 views

Is it possible to use the GPT-2 model for time-series data prediction?

Is it possible and how trivial (or not) might it be (if possible) to retrain GPT-2 on time-series data instead of text?
1
vote
3answers
88 views

How can neural networks be used to generate rather than classify?

In my experience with Neural Nets, I have only used them to take input vectors and return binary output. But, here in a video, https://youtu.be/ajGgd9Ld-Wc?t=214, Kai Fu Lee, renowned AI Expert shows ...
1
vote
0answers
33 views

Finetuning GPT-2 twice for particular style of writing on a particular topic

Sorry if this is a stupid question. I'm just starting out in ML and am working with gpt-2 for text generation. My situation is that I have to generate text in a particular field for eg. family ...
1
vote
0answers
19 views

Why hasn't deep learning been used for word level alignment?

I've been exploring word-level alignments tools such as MGIZA and it seems to me that there hasn't been any new tool for this problem. Are neural networks not suitable to solve this problem or simply ...
1
vote
2answers
35 views

Does it matter if it's a bot or a human generating text? Doesn't it come down to the content?

It was noted today that automated text generation is advancing at a rapid pace, potentially accelerating. As bots become more and more capable of passing turing tests, especially in single iterations,...
2
votes
0answers
22 views

How to train a transformer text-to-text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
0
votes
0answers
11 views

How does transformer network learn to decide in a single step?

I read this In Google AI blog: link: https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html S1 : " the Transformer can learn to immediately attend to the word “river” and make this ...
1
vote
2answers
50 views

Why is embedding important in NLP, and how does autoencoder work?

People say embedding is necessary in NLP because if using just the word indices, the efficiency is not high as similar words are supposed to be related to each other. However, I still don't truly get ...
0
votes
1answer
20 views

How to change this RNN text classification code to become text generation code?

I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T ...
2
votes
1answer
52 views

What is the best way to find the similarities between two text documents?

I would like to develop a platform in which people will write text and upload images. I am going to use Google API to classify the text and extract from the image all kinds of metadata. In the end, I ...
2
votes
0answers
27 views

What is the State-of-the-Art open source Voice Cloning tool right now?

I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning),...
2
votes
0answers
16 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...