Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [natural-language-processing]

Natural language processing is the computer representation and manipulation of human language.

1
vote
1answer
39 views

Calculation of GPU memory consumption on softmax layer doesn't match with the empirical result

I'm training a language model with 5000 vocabularies using a single M60 GPU (w/ actually usable memory about 7.5G). The number ...
0
votes
1answer
49 views

How to make a chatbot with NLP and Neural

I would like to have a chat to talk on a personal plan enough to pass the Turing test. I got to study some natural language processing (NLP) but some that usually hide a lot of mechanical, so I think ...
2
votes
1answer
41 views

How to build a commercial image captioning system?

Image Captioning is a hot research topic in the AI community. There are considerable image captioning models for research usage such as NIC, Neural Talk 2 etc. But can these research models be used ...
3
votes
1answer
26 views

How to parse conjunctions in Natural Language Processing in Python

Is there an accepted way in NLP to parse conjunctions (and/or) in a sentence? Disclaimer: I'm not sure if this is technically a computer science or AI problem. For the below example, how would I ...
2
votes
2answers
79 views

How does the 'Lorum ispum' generator work?

In the domain of natural language processing, a textgenerator is able to produce pseudo random output. The most famous one is SCIgen which was used to generate fake-science papers. The inner working ...
3
votes
2answers
54 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network based models for English and German. And I specifically mean language model in its standard sense. I ...
1
vote
0answers
24 views

Can semantic paraphrasing be used for a workflow management system?

The term paraphrasing is used for converting input text into output text with small modifications on the semantic level. Paraphrasing is used by managers to distribute work items to employees. It is a ...
3
votes
2answers
55 views

Software that understands hometask in mathematics with the hometask being a text

What software can understand the following task: "A big cat needs 4 days to catch all the mice and a small cat needs 12 days. How many days need the both if they catch mice together?"?
1
vote
0answers
15 views

Sequence to sequence machine learning / NMT - converting numbers into words

I want to do some sequence to sequence modelling on source data that looks like this: /-0.013428/-0.124969/-0.13435/0.008087/-0.269241/-0.36849/ with target data ...
3
votes
2answers
68 views

How can I train model to extract custom entities from text?

I have a 100-150 words text and I want to extract particular information like location, product type, dates, specifications and price. Suppose if I arrange a training data which has a text as input ...
5
votes
2answers
76 views

Can you teach an AI through sentences?

If you taught an AI to understand sentences through usual neural network techniques. Then could you being to teach it things with sentences such as "ants are small", "the sky is blue". i.e. if you ...
3
votes
0answers
44 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron ( MLP )branch ...
2
votes
1answer
25 views

Recognising common short forms of appliances used daily

I'm working on a chatbot for homes and would need to understand the appliance the user is talking about. Suppose the command is: Turn on the air conditioner. and another variant could be Turn on the ...
0
votes
0answers
28 views

NLP for semantic retrieval

I want to use NLP for semantic retrieval. I am new to this field. Can anyone suggest toolkit or tutorial to learn and implement NLP from scratch.
1
vote
0answers
54 views

Sentence classification and named identity detection with automatic retraining

I am learning AI and trying out my first real life AI application. What I am trying to do is taking as an input various sentences, and then classifying the sentences into one of X number of categories ...
4
votes
1answer
44 views

Is 'job title classification' rather a problem of NLP or machine learning?

first of all I want to specify the data available and what needs to be achieved: I have a huge amount of vacancies (in the millions). The information about the job title and the job description of ...
1
vote
0answers
13 views

Would Capsule Neworks and Topic Modeling used together make sense?

This is actually something I have been researching a bit on my own. Most movie scripts can be structurally analysed by using writing theory such as Dramatica. Dramatica is based upon hierarchy of ...
2
votes
0answers
28 views

Extracting referenced documents

I'm looking to write an AI that will be able to extract in text references from standards documents to assist human research. My use case is extracting the identifying numbers, for example, "AR 25-2",...
2
votes
1answer
86 views

Can AI solve jumbled words?

My question is that is there any general idea on how humans solve jumbled words? I know many people will say we match it against a commonly used words checklist mentally, but it is kind of vague. Is ...
1
vote
1answer
42 views

What are the internal concepts incorporated in IBM's Watson platform?

IBM's Watson acts as a template for developing chat-bots with ease (without coding), but what are the methodologies and concepts that have been used to build it?
5
votes
1answer
66 views

What algorithms does stackoverflow use for classifying duplicate questions?

Can I get details about the algorithms used for classifying questions in stackoverflow ("Questions that may already have your answer"). Most of the suggestions I get are nowhere related to the ...
2
votes
2answers
73 views

Question classification according to chapters

I have a corpus, say an instruction manual. The text in this manual is grouped into chapters and each chapter is split up into sections. For example, Chapter 1/Section 1, Chapter 1/Section 2 and so on....
2
votes
3answers
192 views

Is there something like OpenCV for voice recognition & NLP?

As you know OpenCV is a big great open-source library for image recognition and machine vision(and may further purposes like computer graphics, etc). Is there similar library in sound field(Voice ...
2
votes
1answer
48 views

Implementing AI/ML in customer service

I am working on a task where I am required to automate the customer service request channel. The process is quite typical. A customer queries about a product via email, the person on the front channel ...
2
votes
1answer
39 views

What existing tools are out there for creating chatbots from a plain text corpus and without having question, answer pairs?

I know some people will try to crucify me for asking such question here, since it's too generic, but ill give it a shot. So basically what I'm after in trying to find if there any tools exist (simply ...
2
votes
2answers
111 views

What is the most effective way to learn natural language processing online?

There are many books, courses, etc. out there, but not sure which path to take. So what would be the most effective way (shortest) to learn natural language processing online? p.s. I mean learning ...
2
votes
1answer
76 views

How can I build an AI with NLP that reads and understands documents?

I have to read a lot of papers, and I thought that I can use an A.I. to read them and summarize them. Maybe find one that can understand what the papers are talking about it seems a lot to ask. I ...
1
vote
1answer
28 views

Are there any references of nlp/text mining techniques for Identifying the theme of News headlines

I am looking to extract the central theme of news headline using NLP/ Text-mining, any references in this direction is of great help. For example: Inputs: BRIEF-Dynasil Corporation Of America ...
0
votes
1answer
232 views

How can I create my own Google duplex?

I am trying to create my own variant of Google duplex however, it won't make calls but just have a real time conversation. My question is, where and how to start? How do I train my model with real ...
1
vote
1answer
41 views

Recommended Modelling Technique for Influencer Marketing Scenario

I have an approximately 90,000 row dataset that has information of social media profiles which has columns for biography, follower count, language spoken, name, username and the label (to identify ...
1
vote
0answers
24 views

Number of utterances

Currently I am working with two large speech databases and I was asked to build one subset from each one in order to get two representative subsets with the same number of utterances. My question is: ...
2
votes
1answer
29 views

Do individual dimensions in vector space have meaning?

Word2vec assigns an N-dimensional vector (a dimensional reduction, really) to given words. It turns out that, at least with a number of canonical examples, vector arithmetic seems to work intuitively. ...
1
vote
0answers
64 views

Optimizing the number of embedding dimensions for word2vec

The choice of the number of embedding dimensions (N) seems to generally be determined empirically (i.e. trial-and-error) for a given corpus and use case. Typical ranges used seem to be anywhere from ...
5
votes
3answers
78 views

Sentiment Analysis

Can we define the feeling of the human through conversations with an AI? Something like a "confessional," disregarding human possibilities to lie. Below, I have the categories joyful, sadness, anger, ...
1
vote
0answers
20 views

Commercial API Q: is there an api for converting vision tags into a caption?

There are many machine learning api for scanning images but they just return a bunch of tags. https://azure.microsoft.com/en-gb/services/cognitive-services/computer-vision/ ...
0
votes
0answers
22 views

Did anyone model NLP system on top of text summarization problem?

When you read a big book, I think humans constantly summarize or model the content of the book to the point where they have read and understand the upcoming text with that summarization as a context. ...
3
votes
1answer
39 views

What are development tests used for?

This is a theoretical question. I am newbie to artificial intelligence and machine learning, and the more I read the more I like this. So far, I have been reading about evaluation of language models (...
2
votes
0answers
55 views

NLP proved against US legal texts

I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts. Obviously some training would need to be done, but I've found little ...
3
votes
2answers
141 views

Two chatbots - One teaches another

I am seeking the information for this kind of chatbot architecture : There are two chatbots. One plays the role of teacher, and another is a student who is learning. The goal is to test the student's ...
0
votes
2answers
55 views

What is easier or more efficient to summarize voice or text? [DP/RN]

If possible consider the relationship between implementation difficulty and accuracy in voice examples or simply chat conversations. And currently, what are the directions on algorithms like Deep ...
4
votes
2answers
54 views

Represent sense/meaning of sentences mathematically

I am absolutely new in AI area. I would like to know how to mathematically/logically represent the sense of sentences like: ...
1
vote
0answers
55 views

Analogies and similarity

The more I think about machine learning the more I realize the importance of finding similarities by using analogies as a way of learning. If I want to categorize words into hierarchical tree this ...
5
votes
1answer
76 views

Methods to tell if a question can be answered from a paragraph

I'm working on a project related to machine Q&A, using the SQuAD dataset. I've implemented a neural-net solution for finding answers in the provided context paragraph, but the system (obviously) ...
1
vote
1answer
58 views

How to find the subject in a text?

I often develop bots and i need to understand what some people are saying. Examples: - I want an apple - I want an a p p l e How to understand the subject (apple)? I honestly don't know where to ...
0
votes
0answers
250 views

How can I convert Natural Language to Sql query in NodeJS?

I am New to AI and I am trying to convert Natural Language(English) to the SQL query for my application. I have to developed this is NODE JS. I have heard about Open NPL Package but I don't have any ...
0
votes
0answers
88 views

Conv2Conv Autoencoder for text sequences

Training a Conv2Conv autoencoder for the purpose of text clustering, with parameters fixed as following: ...
1
vote
2answers
68 views

Existing programs that find out words with same meanings

I'm wondering if these 2 specific programs already exist and if not how hard would it be to write them: A program that would figure out (by only "reading" large amounts of texts in human language 1 ...
1
vote
0answers
21 views

Matrix to check correctness of word embeddings

I am working on Combined word embedding for multiple languages in a common space. Apart from looking into the embedding through TSNE, is there another way to judge the proficiency of the embedding.
5
votes
2answers
6k views

Sentence similarity in Python

I am working on a problem where I need to determine whether two sentences are similar or not. I implemented a solution using BM25 algorithm and wordnet synsets for determining syntactic & ...
4
votes
1answer
206 views

How could you generate sentences from lists of facts

Lets pretend we had a list of facts (similar to prolog tuples) that define some knowledge about some entities. e.g. doing(clean, data) done(collect, data) todo(train, model) todo(write, paper) What ...