I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned on page 6 chapter 3.4 - this loss function O(theta)
is expressed as marginal loglikelihood objective function. I simply does not understand this. The neural network generates logical expression (query) from some question in natural language. The network is trained using question-answer pairs. One could expect that simple sum of correct-1/incorrect=0 result could be good loss function. But there is strange expression that involves P(l|qi, Ti; theta)
that is not mentioned in the article. What is meant by this P
function? As I understand, then many logical forms l
are generated externally for some question qi
. But further I can not understand this. The mentioned article largely builds on other article http://www.aclweb.org/anthology/P16-1003 from which it borrows some terms and ideas.
It is said that l
is treated as latent variable and P
seems to be some kind of probability. Of course, we should assign the greated probability to the right logical form l
, but where can I find this assignment. Does training/supervision data should contain this probability function for training/supervision data?