3
GPT-2 is a close copy of the basic transformer architecture.
GPT-2 does not require the encoder part of the original transformer architecture as it is decoder-only, and there are no encoder attention blocks, so the decoder is equivalent to the encoder, except for the MASKING in the multi-head attention block, the decoder is only allowed to glean information ...
2
There isn't, really. Natural language is way more complex and irregular than algebra, which is far more formalised and unambiguous.
So far, in NLP, most success/progress has been made in little toy domains, which exclude most of the complexities of real life, including many ambiguities.
When you say the rules of algebra are somewhat like grammar, then that ...
1
Both ways are valid. It depends on what you want from the model and expect from the data. Generally though I would use 1 assumption and stick with it (unless there was a specific reason not to), so I would use all lines for test if training done that way, and same for majority.
Also note if you ever get more than 3 people, you can choose to do a variance ...
1
One of the ways to ask if this two problmes are related is to ask, could we solve math/algebra equations with NLP approaches, and the answer is yes, it's an absolutely valid idea and it was approached by many researchers.
For example in the "Deep learning for symbolic mathemathics" paper by facebook researchers, the NLP-based approach was used to ...
Only top voted, non community-wiki answers of a minimum length are eligible
