I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned on page 6 chapter 3.4 - this loss function O(theta) is expressed as marginal loglikelihood objective function. I simply does not understand this. The neural network generates logical expression (query) from some question in natural language. The network is trained using question-answer pairs. One could expect that simple sum of correct-1/incorrect=0 result could be good loss function. But there is strange expression that involves P(l|qi, Ti; theta) that is not mentioned in the article. What is meant by this P function? As I understand, then many logical forms l are generated externally for some question qi. But further I can not understand this. The mentioned article largely builds on other article http://www.aclweb.org/anthology/P16-1003 from which it borrows some terms and ideas.

It is said that l is treated as latent variable and P seems to be some kind of probability. Of course, we should assign the greated probability to the right logical form l, but where can I find this assignment. Does training/supervision data should contain this probability function for training/supervision data?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.