All Questions
Tagged with bert language-model
7 questions
0
votes
1
answer
70
views
How is the bidirectional context achieved in BERT?
I have read the paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" by Jacob Devlin et al. (2018) and "Improving Language Understanding by ...
- 3
0
votes
2
answers
832
views
How big the context can be using HuggingFace models?
I'm new on AI, Neural Networks, ChatBots and all this ecosystem. I'm trying to use a classical example of pre-trained models, more specifically ...
- 103
0
votes
2
answers
1k
views
How to combine pretrained language models with additional feature set?
Are there any techniques to combine a feature set (other than the text itself) with pretrained language models.
Let's say I have a random NLP task that tries to predict a binary class label based on e....
- 101
1
vote
1
answer
2k
views
Is there a way to provide multiple masks to BERT in MLM task?
I'm facing a situation where I've to fetch probabilities from BERT MLM for multiple words in a single sentence.
...
- 317
7
votes
1
answer
1k
views
How to use BERT as a multi-purpose conversational AI?
I'm looking to make an NLP model that can achieve a dual purpose. One purpose is that it can hold interesting conversations (conversational AI), and another being that it can do intent classification ...
- 323
2
votes
0
answers
80
views
How can I generate a document from a single word using GPT or BERT?
I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it.
I came across language models GPT,...
- 161
5
votes
2
answers
2k
views
Where can I find pre-trained language models in English and German? [closed]
Where can I find (more) pre-trained language models? I am especially interested in neural network-based models for English and German.
I am aware only of Language Model on One Billion Word Benchmark ...
- 161