2
$\begingroup$

I don't know anything about ML or NLP, but I was asked by someone to create brand new statutes (written laws) that resemble the ones currently in effect in my country. I have already gathered the laws, and have 5000 html files now, one per law.

The average size of each html file is 49 kB. The entire corpus is 300 MB.

I have two alternative goals (doing both would be perfect of course):

  • Generate a new, complete HTML file, that would imitate the 5000 existing ones (it would typically have 1 big heading at the top, sub-headings, articles with their own title and number, etc.)

  • Generate sentences that sound as if they could be found in a typical law (the laws are written in French)

Is any of those goals feasible, with such a small corpus (~300 MB total)?

Should I try and fine-tune an existing model (but in that case, wouldn't the small size of my corpus be a problem? Wouldn't it be "drowned out" in the rest of the training data?), or should I create one from scratch?

I've tried following guides on huggingface, but between the obsolete files, the undocumented flags and my general lack of knowledge of the subject, I'm completely lost.

Thanks in advance.

BTW, if you want to take a peek at the data, there it is: https://github.com/Biganon/rs/

Improve this question
$\endgroup$
3
  • $\begingroup$ "I don't know anything about ML or NLP" --> why should u take the task, btw, text generation is just for fun, won't ever be law (must be exact, precise, no bad logics) $\endgroup$
    – Dan D.
    Mar 24, 2021 at 10:59
  • 2
    $\begingroup$ @datdinhquoc I took the task because I'm the only teaching assistant in the law faculty who is at least slightly comfortable around computers, and I thought it would be fun. I am not required to fulfill the task, I just thought it would be interesting. Also, we don't want to produce law that will be used, we want to use this as an experiment, precisely to show the limits of the system. I'm still hoping someone can point me in the right direction. $\endgroup$
    – Biganon
    Mar 26, 2021 at 11:26
  • $\begingroup$ Why does it need to be in HTML? Why not simply the raw texts of the statutes themselves. HTML formatting introduces extra complexity, and is the same in every file: it does not to be learned. $\endgroup$
    – Vladislav Gladkikh
    Mar 31, 2021 at 2:55

1 Answer 1

Reset to default
1
$\begingroup$

You could try to train a recurrent neural net on char level. Basically, you took GRU or LSTM and use a sequence of characters, not tags or words. In the blogpost "The Unreasonable Effectiveness of Recurrent Neural Networks" there are examples for Shakespeare, Linux source code on C and for papers in latex code, and the results are quite valid, produced from similar to your size training sets. The good thing in case of html pages is that modern browsers are very good at handling a slightly broken HTML, so this approach could work for both of your tasks.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .