3
$\begingroup$

What will happen if to train an LLM on taking integrals and solving equations? The process of mathematical education can be absolutely automated by a computer algebra system because the verification is easy.

Is it possible that LLM will gan the ability to take integrals and simplify expressions better than the computer algebra system itself?

$\endgroup$
9
  • 2
    $\begingroup$ they struggle with multiplications... $\endgroup$
    – Alberto
    Aug 10, 2023 at 19:50
  • $\begingroup$ "if i train a LLM on pairs of prime numbers and their product will it learn to factorize faster than any other algorithm?" $\endgroup$
    – Alberto
    Aug 10, 2023 at 19:51
  • $\begingroup$ @AlbertoSinigaglia I think, this needs to be tested. $\endgroup$
    – Anixx
    Aug 10, 2023 at 20:06
  • $\begingroup$ nope, you cannot learn to do something by strictly imitating.. at most you can generalize, but you cannot extrapolate... if you want to beat human you first need to extrapolate, and then maybe some RL $\endgroup$
    – Alberto
    Aug 10, 2023 at 22:41
  • 1
    $\begingroup$ Please see this related question: ai.stackexchange.com/questions/25210/… $\endgroup$ Aug 11, 2023 at 8:42

3 Answers 3

5
$\begingroup$

If you're interested in research on using LLMs for arithmetic tasks, Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks is a nice recent (May 2023) article.

If you're interested in research on using LLMs for more advanced mathematics, MiniF2F is a common benchmark. Check out the papers linked from the leaderboard.

If you're looking for LLM models or datasets for mathematics, https://huggingface.co/hoskinson-center and https://leandojo.org/ are good starting points.

$\endgroup$
3
$\begingroup$

Depends what you mean by 'better' (in what sense?).

Nevertheless, though it is possible that an LLM can solve algebra, in general it shall not offer any benefit than a computer algebra system (again depends on what is 'better', and assuming a fair comparison).

This is because in doing so an LLM needs to deal with both the language and the math, while computer algebra system only needs to work on the math. It is like a top human runner and a vehicle - in general vehicle moves faster because this is what they are crafted solely for.

And a few words on the comments - finding rhymes is NOT like solving equations, because no rhymes is wrong (even if it sounds terrible, but that's subjective); but there is straight wrong in solving equations.

On the other hand, there are efforts to discover and improve algorithms, most notably Google's AlphaTensor. However it has nothing to do with LLM, but a lot to do with RL.

$\endgroup$
5
  • $\begingroup$ One can define good and incorrect rhymes, this is not subjective, this is measurable. English poetry rarely has perfect rhymes, but in Russian poetry it is quite common for poems to only have absolutely perfect rhymes. LLMs cannot make Russian poetry yet, though. When they will be able, it would be a sensation. $\endgroup$
    – Anixx
    Aug 11, 2023 at 7:13
  • $\begingroup$ Well, you could have specified it. And make sure this is true for all existing languages. $\endgroup$
    – lpounng
    Aug 11, 2023 at 7:17
  • $\begingroup$ Besides, as a Go player myself, I am not arguing about whether a computer can perform better than human; we are talking about whether LLM can solve algebra better than a machine specially crafted for solving algebra, under fair assumptions. $\endgroup$
    – lpounng
    Aug 11, 2023 at 7:25
  • $\begingroup$ But people take integrals better than CAS can. $\endgroup$
    – Anixx
    Aug 11, 2023 at 11:05
  • $\begingroup$ 1. Maybe, but we are comparing LLM and CAS, not CAS and human. 2. Once again, please formally define what 'better' is. 3. Anyway my statement still holds: given fair assumptions (same computing resources, same level of development effort etc.) a machine specially crafted for 1 task usually performs better than multitasking ones. $\endgroup$
    – lpounng
    Aug 14, 2023 at 1:57
1
$\begingroup$

Is it possible that LLM will gan the ability to take integrals and simplify. expressions better than the computer algebra system itself?

Possibly, but not in one run - i.e., if you instruct it and train it to analyse and simplify step by step, while writing this out, and it fits into the context - it is an experiment.

You may also want to STILL use a GAN approach with multiple LLM working on the problem - one writing, others correcting. Which requires a multi-step approach.

Otherwise: no. Math has specific elements that do not really work well with a "forward only" approach of a neural network.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .