Questions tagged [natural-language-processing]

For questions related to natural language processing, which concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

Finetuning GPT-2 twice for particular style of writing on a particular topic

Sorry if this is a stupid question. I'm just starting out in ML and am working with gpt-2 for text generation. My situation is that I have to generate text in a particular field for eg. family ...
8
votes
3answers
5k views

Can BERT be used for sentence generating tasks?

I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words. ...
1
vote
0answers
18 views

Why hasn't deep learning been used for word level alignment?

I've been exploring word-level alignments tools such as MGIZA and it seems to me that there hasn't been any new tool for this problem. Are neural networks not suitable to solve this problem or simply ...
1
vote
1answer
18 views

Convolutional Sequence to Sequence Learning kernel parameters

I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look ...
1
vote
2answers
30 views

Does it matter if it's a bot or a human generating text? Doesn't it come down to the content?

It was noted today that automated text generation is advancing at a rapid pace, potentially accelerating. As bots become more and more capable of passing turing tests, especially in single iterations,...
1
vote
0answers
16 views

How to train a transformer text to text model on counterexamples?

Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples? For example, from the PAWS data set, given the phrases "Although interchangeable, the ...
1
vote
2answers
42 views

Why is embedding important in NLP, and how does autoencoder work?

People say embedding is necessary in NLP because if using just the word indices, the efficiency is not high as similar words are supposed to be related to each other. However, I still don't truly get ...
3
votes
2answers
158 views

Handling emotion in informal text (Hi vs HIIIIII!!!!)?

This is a question related to Neural network to detect "spam"?. I'm wondering how it would be possible to handle the emotion conveyed in text. In informal writing, especially among a ...
7
votes
1answer
441 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
0
votes
1answer
19 views

How to change this RNN text classification code to become text generation code?

I can do text classification with RNN, in which the last output of RNN (rnn_outputs[-1]) is used to matmul with output layer weight and plus bias. That is getting a word (class name) after the last T ...
3
votes
2answers
295 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network based models for English and German. And I specifically mean language model in its standard sense. I ...
11
votes
3answers
865 views

What topologies are largely unexplored in machine learning?

Geometry and AI Matrices, cubes, layers, stacks, and hierarchies are what we could accurately call topologies. Consider topology in this context the higher level geometrical design of a learning ...
4
votes
2answers
126 views

Is there a service that can search for answers to general questions?

I was looking for an API service where I can ask it a general question (for example, when was Einstein born?) and retrieve an answer from the web. Is there any available service to do that? Have ...
0
votes
1answer
41 views

What AI designs are suited for producing title replacements?

Problem: "For a given news article, generate another title for the article if the article is to be published under a different Publication." Which algorithm will be well suited for this? Should I use ...
1
vote
2answers
62 views

What is the difference between automatic transcription and automatic speech recognition?

What is the difference between automatic transcription and automatic speech recognition? Are they the same? Is my following interpretation correct? Automatic transcription: it converts the speech to ...
2
votes
0answers
19 views

What is the State-of-the-Art open source Voice Cloning tool right now?

I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning),...
2
votes
0answers
16 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
1answer
82 views

How do I use neural networks to implement a chatbot?

I don't know anything about neural networks, but I got the information that making a chatbot with a neural network is very good. Is this really true? What do I need to know in order to build a chat ...
0
votes
1answer
37 views

Preparation of input data

Tell me why my val_acc is always the same and how to solve this problem? I saw several topics on the Internet specifically on this problem but they did not help me (for example, use SGD with different ...
1
vote
1answer
35 views

Can AI help summarize article or abstract sentence keyword?

I'm wondering if AI now can help us abstract summary or general idea of long article, for example novel or historical stories, or abstract most important keyword from sentence; Would you please tell ...
1
vote
2answers
75 views

Which algorithm should I use to map an input sentence to an output sentence?

I am new to NLP realm. If you have an input text "The price of orange has increased" and output text "Increase the production of orange". Can we make our RNN model to predict the output text? Or what ...
1
vote
1answer
193 views

How to make meaningful sentences from a set of words?

I have set of topics generated using LDA and like {code, language, test , write, function}, {class, public, method, string, int} etc and I want to make meaningful sentence/sentences from these words ...
6
votes
2answers
202 views

How would an AI learn idiomatic phrases in a natural language?

After an AI goes through the process described in How would an AI learn language?, an AI knows the grammar of a language through the process of grammar induction. They can speak the language, but they ...
0
votes
2answers
96 views

Content analysis tools

We want to figure out the connection between people, based on their speech. Assume, that a conversation is a poetry with lines belongs to characters. There are a lot of poetries and the lines are ...
11
votes
3answers
401 views

How would an AI learn language?

I was think about AIs and how they would work, when I realised that I couldn't think of a way that an AI could be taught language. A child tends to learn language through associations of language and ...
-1
votes
1answer
119 views

Can a neural net learn to read?

I am a student of last year of computer engineering and lately I have been very interested in AI. Fields such as ML and DL seem very disruptive to me. A few months ago I saw an interview of Bill ...
3
votes
1answer
93 views

Is it possible to verify SOP of call center conversation using AI?

As we know in call centers there are certain SOP(standard operating procedure), for example few are below, Agent greeted customer Agent verified the validity of customer before providing sensitive ...
2
votes
1answer
44 views

Why do you need to retrain GPT-2?

I'm following this tutorial, and I wonder why is there a train-step - why is it necessary? I thought the whole idea of GPT-2 is that you do not need to train it on specific text domain, as it's ...
3
votes
2answers
125 views

Software that understands hometask in mathematics with the hometask being a text

What software can understand the following task: "A big cat needs 4 days to catch all the mice and a small cat needs 12 days. How many days need the both if they catch mice together?"?
1
vote
2answers
123 views

How can I make meaningful English sentences from given set of words?

I have a set of topics and each topic consists of a set of words. I want to make meaningful English sentences from these words. Each topic consist of 5 to 10 words and these words are relevant to each ...
4
votes
4answers
469 views

Can Machine Learning be applied to decipher the script of lost ancient languages?

Can Machine Learning be applied to decipher the script of lost ancient languages? (Languages that many years ago were being used, but currently are not used in the human societies and have been ...
3
votes
1answer
52 views

Techniques for sentence transformation and consistency

I'm looking for NLP techniques to transform sentences without affecting their meaning, and measures of textual consistency. For example, techniques that could transform active voice into passive voice,...
2
votes
2answers
57 views

Chatbot using Machine Learning vs Programming

I am currently building a chatbot. What I have done so far is, collected possible questions/training data/files and created a model out of it using Apache OpenNLP; the model is able to predict all the ...
2
votes
1answer
51 views

How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?

This is related to my earlier question, which I'm trying to break down into parts (this being the first). I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. ...
6
votes
2answers
291 views
0
votes
0answers
19 views

Understanding how continuous bag of words method learns embedded representations

I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding: If we have a vocabulary $V$, the naive way to ...
2
votes
2answers
111 views

Is there a way to understand the type of a sentence?

I am a beginner, just started studying around NLP, specifically various language models. So far, my understanding is that - the goal is to understand/produce natural language. So far the methods I ...
2
votes
2answers
36 views

Can ELMO embeddings be used to find the n most similar sentences?

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a ...
0
votes
1answer
33 views

How to use pretrained checkpoints of BERT model on semantic text similarity task?

I am unaware to use the derived checkpoints from pre-trained BERT model for the task of semantic text similarity. ...
0
votes
1answer
44 views

1D convolutions, word2vec, and $n$-grams

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams. Question 1 Why would adding all the possible $\...
1
vote
1answer
28 views

Sentiment Analysis and Logistic Regression

Suppose we want to classify a review as good ($1$) or bad ($0$). We have a training data set of $10,000$ reviews. Also suppose we have a vocabulary of $100,000$ words $w_1, \dots, w_{100,000}$. So the ...
1
vote
2answers
40 views

What is “Word Sense Disambiguation”?

I recently came across this article which cites a paper which apparently won outstanding paper in ACL 2019. The theme is that it solved a longstanding problem called Word Sense Disambiguation. What ...
1
vote
3answers
43 views

Why do language models place less importance on punctuation?

I have very outdated idea about how NLP tasks are carried out by normal RNN's, LSTM's/GRU's, word2vec, etc to basically generate some hidden form of the sentence understood by the machine. One of the ...
2
votes
1answer
78 views

Anomaly Detection in distributed system using generated log file

I am developing an AI tool for anomaly detection in a distributed system.  The system supports an interface that combines several individual logs into a single log file generating approx. 7000 entries/...
2
votes
0answers
39 views

Grouped Text classification

I have thousands groups of paragraphs and I need to classify these paragraphs. The problem is that I need to classify each paragraph based on other paragraphs in the group! For example, a paragraph ...
2
votes
0answers
76 views

How do the Sine and Cosine functions encode position in the “Attention is All You Need” paper?

After going through both the "Illustrated Transformer" and "Annotated Transformer" blog posts, I still don't understand how the sinusoidal encodings are representing the position of elements in the ...
0
votes
2answers
236 views

Use BERT to answer a FAQ with semantic similarity

I have been looking for BERT for many tasks and I would like to compare performance to answer a FAQ using BERT semantic similarity and BERT Q/A. However, I'm not sure it is a good idea to use ...
3
votes
0answers
370 views

RL to generate sentences

I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. Eg: Input: capital, Paris, ...
5
votes
4answers
515 views

Is 'job title classification' rather a problem of NLP or machine learning?

first of all I want to specify the data available and what needs to be achieved: I have a huge amount of vacancies (in the millions). The information about the job title and the job description of ...
1
vote
0answers
6 views

How can we use Dependency Parsers for Negation detection

I am building a negation detection system. How to use dependency parsers for the same. I am using SPACY for dependency parser