How would an AI learn language?

Question

I was think about AIs and how they would work, when I realised that I couldn't think of a way that an AI could be taught language. A child tends to learn language through associations of language and pictures to an object (e.g., people saying the word "dog" while around a dog, and later realising that people say "a dog" and "a car" and learn what "a" means, etc.). However, a text based AI couldn't use this method to learn, as they wouldn't have access to any sort of input device.

The only way I could come up with is programming in every word, and rule, in the English language (or whatever language it is meant to 'speak' in), however that would, potentially, take years to do.

Does anyone have any ideas on how this could be done? Or if it has been done already, if so how?

By the way, in this context, I am using AI to mean an Artificial Intelligence system with near-human intelligence, and no prior knowledge of language.

Answer 1

The general research area is known as grammar induction.

It is generally framed as a supervised learning problem, with the input presented as raw text, and the desired output the corresponding parse tree. The training set often consists of both positive and negative examples.

There is no single best method for achieving this, but some of the techniques that have been used to date include:

• Bayesian approaches
• Genetic Algorithms
• Genetic Programming
• Blackboard Architectures
• The UpWrite Predictor

Answer 2

The umbrella term for your problem is called natural language processing (NLP) -- a topic under artificial intelligence.

There are many subtopics to this field including language semantics, grammatical analysis, parts of speech tagging, domain specific context analysis, etc.

Answer 3

Just for the sake of completeness, I'll point out that Recurrent Neural Nets (i.e. neural nets with backwards connections) are frequently used for for Natural Language Processing (NLP). This includes variants like Bidirectional, Jordan and Elman Networks. Long Short-Term Memory (LSTM) is a more sophisticated neural net algorithm which can accomplish the same time and sequence-based tasks, but which can leverage standard learning methods like backprop since it doesn't suffer from the "vanishing gradient problem." This is because LSTMs have been brilliantly engineered as "perfect integrators," which makes it a lot easier to calculate the error gradients etc. over long periods of time. In contrast, learning with RNNs is still not theoretically well-grounded and is difficult to calculate through existing methods like Backpropagation Through Time (BPTT). In Time Delay Neural Networks (TDNNs), the idea is to add new neurons and connections with each new training example across a stretch of time or training sequence; unfortunately, this places a practical limitation on how many examples you can feed into the net before the size of the network gets out of hand or it starts forgetting, just as with RNNs. LSTMs have much longer memories (especially when augmented with Neural Turing Machines) so that'd be my first choice, assuming I wanted to use neural nets for NLP purposes. My knowledge of the subject is limited though (I'm still trying to learn the ropes) so there may be other important neural net algorithms I'm overlooking...