I am researching Natural Language Processing (NLP) to develop a NL Question Answering system. The answering part is already done. So processing the question remains, along with the questions regarding the algorithms.

The final product should allow the user to ask a question in NL; the question then gets translated to an MDX query, which generates a script regarding dimensions of the cube.

How can I translate a natural language question to an MDX query? The outcome of question is in form of a calculation. E.g. ‘How many declarations were done by employee1?’ or ‘Give me the quantities for Sales’

  • 2
    $\begingroup$ First go through this question DataScience $\endgroup$ – quintumnia Feb 4 '17 at 14:08
  • $\begingroup$ I'm voting to close this question as off-topic (see scope defined in help center). At present it would rather migrate to StackOverflow to get an answer. $\endgroup$ – Eric Platon Feb 14 '17 at 3:22

You can use a component library which can help you to implement Natural language query builder in your application( the question part ) called Open Natural Language Processing Package , so you can definitely develop a module, by using existing modules of OpenNLP such as entity extraction, chunking and parsing.

According to wikipedia source ; it points out that as of 2001, Q&A applications typically includes "a question classifier module which determines the type of question and the type of answer," so "a multiagent question and answering architecture has been proposed, where each domain [or variable] is represented by an agent which tries to answer questions, taking into account its specific knowledge."

But it still need some effort to build a NLgenerators inline with databases , for the answer query and also is the link to help you on how you can work with Compositional Semantic Parsing on Semi-Structured Tables

Hope this can give you some insight.

| improve this answer | |
  • 1
    $\begingroup$ Thank you for linking me to the other topic, and answering it here as well! You brought me on many ideas, and cleared things out! $\endgroup$ – lilienfa Feb 6 '17 at 9:50
  • $\begingroup$ @lilienfa Pleasure; but first tour the community and your will soon get feedback from other professionals as well. $\endgroup$ – quintumnia Feb 6 '17 at 10:07
  • $\begingroup$ @quintumnioa, thank you for the link, I understand the platform better now :) . I might still have a small question for you, regarding parsing the question. I do not know if you are familiar with parsing algorithms, but if you do, for the parsing, would you use CKY, Earley or chart parsing algorithm? And why would you choose that specific one? $\endgroup$ – lilienfa Feb 6 '17 at 10:13
  • $\begingroup$ Probably I would chose Earley parsing algorithm; in my context Earley parsers are awesome, because they will parse anything you give them. depending on the algorithm specified, popular parsers such as lex/yacc, flex/bison, Jison, PEGjs, and Antlr will break depending on the grammar you give it. And by break, I mean infinite loops caused by left recursion, crashes, or stubborn refusals to compile because of a “shift-reduce error”. $\endgroup$ – quintumnia Feb 6 '17 at 10:48
  • $\begingroup$ Thank you! Indeed, just read a comparison. And Earley outperforms CKY, is faster, goal-driven, context free. Everything I need for my product. Thank you so much! Im particularly excited about this project right now :D $\endgroup$ – lilienfa Feb 6 '17 at 10:50

This is a hard problem to solve, and the best approach depends very much on the scope of your task. If you have a small database table with a limited number of columns, you might get away with some basic pattern matching techniques. If it is more complex than that, you might have to do a full-scale syntactic analysis of the question. This also depends on the variations of possible question types.

Assuming a limited set of variables and variants, you could set up something like:

How many X did Y produce/How many X were done by Y/What is the number of X for Y

where you have two variables to fill from the pattern, which you then use in your query:

select sum(X) where producer == Y

(Or whatever format your query has).

The advantage of this is that you don't need to be a linguistics expert to maintain/expand the system, and you can just add more patterns to it if necessary. You might have to map some terms onto synonyms to get the right column headings/labels out of it. But this approach is not very hard to implement, and you should have a basic system up and running fairly quickly. You then have to see/test what questions your users are asking, and expand the pattern inventory accordingly.

The disadvantage is that you might end up with a long list of patterns, and there could be some which are conflicting, ie the same pattern with different variables will ask for a different kind of result. If that turns out to be a problem, you might have to look for a more powerful approach.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.