0
$\begingroup$

I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.

In the project, I have been given a data set with (rather small) 30.000 text documents, which are labeled with 0 and 1. I want to train and evaluate (with respect to accuracy) a BERT and XLNet model.

Can you give me some rough estimates for the following questions?:

  1. How much computing power do I need for this task, i.e. can I simply use my private laptop for this or do I need a special CPU/GPU for it?
  2. So far, I just worked with classical machine learning models (e.g. random forests, SVMs, etc.). I am not experienced deep learning architectures yet. How difficult would it be to implement a BERT oder XLNet model with my own data set, having no experience with BERT oder XLNet yet? I.e. how much code would it be that I have to develop by myself? And would I need a deep understanding for it or would be sufficient to follow an online tutorial and basically copy the code from there? Many thanks.
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$
  1. You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’t try it on a CPU. It will “work” but you’re gonna have a bad time.
  2. Try the Huggingface Transformers library as your implementation. It’s well documented and straightforward and includes both models.

*assuming an Nvida GPU or something compatible with CUDA. Things are rather hairier on Apple hardware. But you can always grab a cloud VM for a few hours

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged .