Does anyone know of research involving the GPT models to learn not only regular texts, but also learn from physics books with the equations written in latex format?

My intuition is that the model might learn the rules relating equations and deductions, as they can learn statistically what correlates with what. I understand that the results can also be a little nonsensical, like the sometimes surreal paragraphs written by these models.

Have there been any attempts to do this?

  • 3
    $\begingroup$ It's very unlikely that any machine learning model can really understand physics (in a "complete" way that a human can). Maybe you can use these models to find correlations between mathematical symbols and descriptions, but a full or complete understanding of a difficult subject like physics is unlikely to be accomplished with a single model. However, note that neural networks (or deep learning systems) have already been used to solve math problems (at least, people have tried to do this). See, for example, this question. $\endgroup$ – nbro Sep 4 '20 at 14:39
  • 2
    $\begingroup$ There is an interesting paper that tests GPT, BERT and ELMo to "show that the Transformer can be mathematically interpreted as a numerical Ordinary Differential Equation (ODE) solver for a convection-diffusion equation in a multi-particle dynamic system". See web.stanford.edu/~yplu/pub/TransformerODE.pdf $\endgroup$ – Brian O'Donnell Sep 6 '20 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.