Questions tagged [natural-language-processing]

For questions related to natural language processing, which is concerned with the interactions between computers and human (or natural) languages, in particular how to create programs that process and analyze large amounts of natural language data.

Filter by
Sorted by
Tagged with
2
votes
1answer
86 views

What are the internal concepts incorporated in IBM's Watson platform? [duplicate]

IBM's Watson acts as a template for developing chat-bots with ease (without coding), but what are the methodologies and concepts that have been used to build it?
5
votes
1answer
325 views

What algorithms does stackoverflow use for classifying duplicate questions?

Can I get details about the algorithms used for classifying questions in stackoverflow ("Questions that may already have your answer"). Most of the suggestions I get are nowhere related to the ...
2
votes
2answers
142 views

Question classification according to chapters

I have a corpus, say an instruction manual. The text in this manual is grouped into chapters and each chapter is split up into sections. For example, Chapter 1/Section 1, Chapter 1/Section 2 and so on....
3
votes
4answers
2k views

Is there something like OpenCV for voice recognition & NLP?

As you know OpenCV is a big great open-source library for image recognition and machine vision(and may further purposes like computer graphics, etc). Is there similar library in sound field(Voice ...
3
votes
1answer
71 views

Implementing AI/ML in customer service

I am working on a task where I am required to automate the customer service request channel. The process is quite typical. A customer queries about a product via email, the person on the front channel ...
3
votes
1answer
76 views

What existing tools are out there for creating chatbots from a plain text corpus and without having question, answer pairs?

I know some people will try to crucify me for asking such question here, since it's too generic, but ill give it a shot. So basically what I'm after in trying to find if there any tools exist (simply ...
3
votes
2answers
106 views

How should the output layer of an LSTM be when the output are word embeddings?

I'm having trouble grasping how to output word embeddings from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
3
votes
3answers
459 views

What is the most effective way to learn natural language processing online?

There are many books, courses, etc. out there, but not sure which path to take. So what would be the most effective way (shortest) to learn natural language processing online? p.s. I mean learning ...
2
votes
1answer
114 views

How can I build an AI with NLP that reads and understands documents?

I have to read a lot of papers, and I thought that I can use an A.I. to read them and summarize them. Maybe find one that can understand what the papers are talking about it seems a lot to ask. I ...
2
votes
1answer
66 views

Are there any references of nlp/text mining techniques for Identifying the theme of News headlines

I am looking to extract the central theme of news headline using NLP/ Text-mining, any references in this direction is of great help. For example: Inputs: BRIEF-Dynasil Corporation Of America ...
0
votes
1answer
508 views

How can I create my own Google duplex?

I am trying to create my own variant of Google duplex however, it won't make calls but just have a real time conversation. My question is, where and how to start? How do I train my model with real ...
2
votes
1answer
52 views

Recommended Modelling Technique for Influencer Marketing Scenario

I have an approximately 90,000 row dataset that has information of social media profiles which has columns for biography, follower count, language spoken, name, username and the label (to identify ...
1
vote
0answers
43 views

Number of utterances

Currently I am working with two large speech databases and I was asked to build one subset from each one in order to get two representative subsets with the same number of utterances. My question is: ...
4
votes
1answer
46 views

Do individual dimensions in vector space have meaning?

Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction). It turns out that, at least with a number of canonical examples, vector ...
-2
votes
1answer
109 views

I need a word database… Any qualities I should look for?

I need a word database to train from. I found a word2vec JS word vector database, but I need a method to teach it which words go in which patterns. Please note that I am not asking to have you ...
4
votes
3answers
136 views

Sentiment Analysis

Can we define the feeling of the human through conversations with an AI? Something like a "confessional," disregarding human possibilities to lie. Below, I have the categories joyful, sadness, anger, ...
5
votes
1answer
893 views

Were the requirements to solve the Turing Test in “AI: A Modern Approach” foreseen by Alan Turing, or backfilled by Peter Norvig and Stuart Russell?

In Section 1.1 of Artificial Intelligence: A Modern Approach, it is stated that a computer which passes the Turing Test would need 4 capabilities, and that these 4 capabilities comprise most of the ...
1
vote
0answers
24 views

Commercial API Q: is there an api for converting vision tags into a caption?

There are many machine learning api for scanning images but they just return a bunch of tags. https://azure.microsoft.com/en-gb/services/cognitive-services/computer-vision/ ...
3
votes
1answer
50 views

What are development tests used for?

This is a theoretical question. I am newbie to artificial intelligence and machine learning, and the more I read the more I like this. So far, I have been reading about evaluation of language models (...
2
votes
0answers
84 views

NLP proved against US legal texts

I'm new to AI development and am looking for a quality algorithm (potentially nlp?) implementation proved against US legal texts. Obviously some training would need to be done, but I've found little ...
3
votes
1answer
988 views

Seq2Seq dialogs predicts only most common words like `you` after couple of epoches

I'm training Seq2Seq model on OpenSubtitles dialogs - Cornell-Movie-Dialogs-Corpus. My work based on the following papers (but currently I'm not implemented Attention yet): Sequence to Sequence ...
2
votes
2answers
245 views

Two chatbots - One teaches another

I am seeking the information for this kind of chatbot architecture : There are two chatbots. One plays the role of teacher, and another is a student who is learning. The goal is to test the student's ...
2
votes
2answers
76 views

What is easier or more efficient to summarize voice or text? [DP/RN]

If possible consider the relationship between implementation difficulty and accuracy in voice examples or simply chat conversations. And currently, what are the directions on algorithms like Deep ...
2
votes
2answers
61 views

Intuition on how word embeddings bring information to a network

How is it that word embedding layer (say word2vec) brings more insights to the network compared to a simple one hot encoded layer? I understand how word embedding carry some semantic meaning, but it ...
3
votes
2answers
66 views

Represent sense/meaning of sentences mathematically

I am absolutely new in AI area. I would like to know how to mathematically/logically represent the sense of sentences like: ...
1
vote
0answers
258 views

Analogies and similarity

The more I think about machine learning the more I realize the importance of finding similarities by using analogies as a way of learning. If I want to categorize words into hierarchical tree this ...
5
votes
1answer
132 views

Methods to tell if a question can be answered from a paragraph

I'm working on a project related to machine Q&A, using the SQuAD dataset. I've implemented a neural-net solution for finding answers in the provided context paragraph, but the system (obviously) ...
1
vote
2answers
318 views

How to find the subject in a text?

I often develop bots and I need to understand what some people are saying. Examples: - I want an apple - I want an a p p l e How do I find the object (apple)? I honestly don't know where to start ...
1
vote
1answer
76 views

How is the word embedding represented in the paper “Recurrent neural network based language model”?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding $w(t)$ is obtained: The reason I wonder ...
1
vote
2answers
90 views

Existing programs that find out words with same meanings

I'm wondering if these 2 specific programs already exist and if not how hard would it be to write them: A program that would figure out (by only "reading" large amounts of texts in human language 1 ...
2
votes
3answers
844 views

How could a chat bot learn synonyms?

I have started to make a chatbot. It has a list of greetings that it understands and responds to with its own list of greetings. How could a bot learn a new greeting or a synonym for a word it ...
2
votes
1answer
184 views

How can I improve this word-prediction AI?

I'm relatively new to AI, and I've tried to create one that "speaks". Here's how it works: 1. Get training data e.g 'Jim ran to the shop to buy candy' 2. The data gets split into overlapping 'chains' ...
3
votes
2answers
96 views

Figure out the meaning of words

Imagine two languages that have only these words: Man = 1, deer = 2, eat = 3, grass = 4 And you would form all sentences possible from these words: ...
0
votes
4answers
929 views

Google Translate: Different translations for the same text

I don't understand why Google Translate translates the same text in different ways. Here is the Wikipedia page of the 1973 film "Enter the Dragon". You can see that its traditional Chinese title is: ...
12
votes
3answers
16k views

How do I compute the structural similarity between sentences?

I am working on a problem where I need to determine whether two sentences are similar or not. I implemented a solution using BM25 algorithm and wordnet synsets for determining syntactic & ...
4
votes
1answer
427 views

How could you generate sentences from lists of facts

Lets pretend we had a list of facts (similar to prolog tuples) that define some knowledge about some entities. e.g. doing(clean, data) done(collect, data) todo(train, model) todo(write, paper) What ...
1
vote
0answers
125 views

Natural language processing with a continuous dependent variable

I have a large number of observations. Each observation contains: dependent variable: a scores ranging from 0 - 100 independent variable: a large article I want to know which words or phrases ...
4
votes
2answers
2k views

Has Capsule Networks or similar systems been used for NLP?

Capsule Networks seem to be a good solution for problems, which make up hierarchical complexity ( ( (eyes, nose, ears -> face); (fingers, nails, palm -> hand) ) -> human). NLP domain is a very clear ...
2
votes
0answers
35 views

What is the “question” when using Dynamic Memory Networks to do part-of-speech tagging and sentiment analysis?

In the paper Ask Me Anything: Dynamic Memory Networks for Natural Language Processing the authors described a Dynamic Memory Network in the context of question answering. Then, they also tested the ...
0
votes
2answers
103 views

Could we code in Natural language once we pass the Turing test?

Technically speaking, could we code in natural language once we pass the Turing test? Would passing the Turing test at least simplify programming languages' syntax?
4
votes
1answer
99 views

Is it possible to verify SOP of call center conversation using AI?

As we know in call centers there are certain SOP(standard operating procedure), for example few are below, Agent greeted customer Agent verified the validity of customer before providing sensitive ...
2
votes
1answer
37 views

Has there been “prescriptive” attempts at language-learning/comprehension in AI?

Is there examples of AI systems being taught grammatical rules, or have such rules built-in to them, in order to recognize commands and the like in natural language? For example, is there a program, ...
3
votes
0answers
127 views

Training RNN's on text: Can you use an ASCII encoding just as well as a one-hot character encoding?

I've mostly seen (e.g. in http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that when training RNN's on text for something like language modeling, the text is usually featurized character-by-...
0
votes
2answers
347 views

Build Conversational AI

I'm looking to build a conversational AI. The goal is to create an AI that is your friend (as opposed to personal assistants that are meant for carrying out commands like "send a text, play that song.....
8
votes
1answer
8k views

Are the dialogs at Sophia's (the robot) appearings scripted?

I talk about the robot from: Hanson Robotics, which was granted the right to citizenship from Saudi Arabia. I have found the following articles: Your new friend is a humanoid robot source: ...
1
vote
2answers
59 views

Consiousness and Language

It appears to me that the thoughts in our heads or at least mine are a result of the language they know. I do not have any thought that is not a consequence of the information that is contained within ...
1
vote
0answers
35 views

Any software to guess idea flow (basic ideas/statements and their derivations) throughout text?

Is there any existing software which analyzes idea flow in the text (or between different texts, e.g scientific articles) and for example visualizes it with some graph?
4
votes
1answer
3k views

Automatic story generator

How can l leverage Artificial Intelligence and Virtual Reality to create Intelligent Automatic Story Generation? My idea is to come up with a system that is empowering the user’s imagination to ...
-2
votes
1answer
44 views

Over-exposure of certain items in content based recommendation engine

I'm working on a content based recommendation engine for ebooks. I create document vectors with 300 features for every ebook using a word2vec model trained on google news and determine recommendations ...
6
votes
4answers
3k views

Is there any open source counterpart to the IBM Watson?

I am looking for something similar to IBM Watson but open source.