5
$\begingroup$

In the original GloVe paper, the authors discuss group theory when coming up with the equation (4). Is it possible that the authors came up with this model, figured out it was good, and then later found out various group theory justifications that justified it? Or was it discovered sequentially as it is described in the paper?

More generally: In AI research, are most things discovered because they work empirically and later justified mathematically, or is it the other way around?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ This question is not only related to AI but research in general (even though you provide the example of GloVe). $\endgroup$
    – nbro
    Dec 12, 2019 at 12:01
  • $\begingroup$ See also this question Are Neural Net architectures accidental discoveries?. $\endgroup$
    – user9947
    Dec 12, 2019 at 12:27

1 Answer 1

Reset to default
1
$\begingroup$

There are definitely approaches that are theory driven (like SVMs), and others where the theory comes after the practice (like a lot of deep neural networks). I think it would be difficult to argue strongly that either direction is more common "in general" within AI, or indeed, within any other branch of science.

The approach that is currently in favor will tend to change over time as well. If we develop a new method that we don't understand, it will tend to attract interest from theorists. If they are able to understand it well, that understanding will likely lead to natural next steps that improve the model. As with other areas of science, the empirical cycle is at work.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .