Questions tagged [language-model]

For questions related to the concept of a language model, which is a probability distribution over sequences of words (for example, of a natural language, such as English).

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

How to pad sequences during training for an encoder decoder model

I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder. However, up until now, I have ...
4
votes
2answers
334 views

How to use BERT as a multi-purpose conversational AI?

I'm looking to make an NLP model that can achieve a dual purpose. One purpose is that it can hold interesting conversations (conversational AI), and another being that it can do intent classification ...
4
votes
3answers
132 views

What would be a good internal language for an AI?

For an AI to represent the world, it would be good if it could translate human sentences into something more precise. We know, for example, that mathematics can be built up from set theory. So ...
4
votes
2answers
79 views

Are embeddings in multi-lingual language models comparable across languages?

Facebook has just pushed out a bigger version of their multi-lingual language model XLM, called XLM-R. My question is: do these kind of multi-lingual models imply, or even ensure, that their ...
2
votes
1answer
36 views

How do you build a language model to predict the contextual similarity between two documents?

How do you build a language model to predict the contextual similarity between two documents?
2
votes
0answers
37 views

How can I generate a document from a single word using GPT or BERT?

I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it. I came across language models GPT,...
4
votes
2answers
738 views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network-based models for English and German. I am aware only of Language Model on One Billion Word Benchmark ...