Questions tagged [language-model]

For questions related to the concept of a language model, which is a probability distribution over sequences of words (for example, of a natural language, such as English).

Filter by
Sorted by
Tagged with
0
votes
1answer
49 views

Are there any good alternatives to an LSTM language model for text generation?

I have a trained LSTM language model and want to use it to generate text. The standard approach for this seems to be: Apply softmax function Take a weighted random choice to determine the next word ...
1
vote
0answers
4 views

POS Tag Frequency and Language Translation

When we translate a text from one language to another, how does the frequency of various POS tags change? So let's say we have a text in English with 10% nouns, 20% adjectives, 15% adverbs, 25% verbs, ...
1
vote
0answers
26 views

How is input defined for a biaxial lstm network for generating music?

I am reading Composing Music With Recurrent Neural Networks by Daniel D. Johnson. But I am really confused about the input passed to this network. If we pass notes of music along the time axis, then ...
1
vote
1answer
30 views

Is there a way to provide multiple masks to BERT in MLM task?

I'm facing a situation where I've to fetch probabilities from BERT MLM for multiple words in a single sentence. ...
1
vote
1answer
26 views

Appropriate metric and approach for natural language generation for small sentences

I am trying to create a language generation model to generate very short sentences/words, like a rapper name generator. The sentences in my dataset are anywhere between 1 word and 15 words (3-155 ...
2
votes
1answer
89 views

Fundamentally, what is a perfect language model?

Suppose that we want to generate a sentence made of words according to language $L$: $$ W_1 W_2 \ldots W_n $$ Question: What is the perfect language model? I ask about perfect because I want to know ...
0
votes
0answers
25 views

Comparing English and Spanish Queries using NLP

I am trying to see whether there are differences in sentences between the topics in 2 languages ie English and Spanish. Eg. (Face masks are mandatory, la mascarilla es obligatoria) but this stretched ...
0
votes
0answers
29 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how can Kneser-Ney Smoothing technique be integrated into a feed-...
1
vote
1answer
28 views

What are the main differences between a language model and a machine translation model?

What are the main differences between a language model and a machine translation model?
1
vote
0answers
29 views

How to pad sequences during training for an encoder decoder model

I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder. However, up until now, I have ...
6
votes
1answer
714 views

How to use BERT as a multi-purpose conversational AI?

I'm looking to make an NLP model that can achieve a dual purpose. One purpose is that it can hold interesting conversations (conversational AI), and another being that it can do intent classification ...
4
votes
3answers
143 views

What would be a good internal language for an AI?

For an AI to represent the world, it would be good if it could translate human sentences into something more precise. We know, for example, that mathematics can be built up from set theory. So ...
4
votes
2answers
90 views

Are embeddings in multi-lingual language models comparable across languages?

Facebook has just pushed out a bigger version of their multi-lingual language model XLM, called XLM-R. My question is: do these kind of multi-lingual models imply, or even ensure, that their ...
2
votes
1answer
50 views

How do you build a language model to predict the contextual similarity between two documents?

How do you build a language model to predict the contextual similarity between two documents?
2
votes
0answers
41 views

How can I generate a document from a single word using GPT or BERT?

I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it. I came across language models GPT,...
4
votes
2answers
1k views

Where can I find pre-trained language models in English and German?

Where can I find (more) pre-trained language models? I am especially interested in neural network-based models for English and German. I am aware only of Language Model on One Billion Word Benchmark ...