I'm working on a project related to machine Q&A, using the SQuAD dataset. I've implemented a neural-net solution for finding answers in the provided context paragraph, but the system (obviously) struggles when given questions that are unanswerable from the context. It usually produces answers that are nonsensical and of the wrong entity type.

Is there any existing research in telling whether or not a question is answerable using the info in a context paragraph? Or whether a generated answer is valid? I considered textual entailment but it doesn't seem to be exactly what I'm looking for (though maybe I'm wrong about that?)

  • $\begingroup$ That isn't an easy problem since you'd need AI capable of making common sense deductions. For example: There was war. Were there any dead people and of what gender and age were they? Yes and they were mostly young men. Or John is older than Peter. Ann is younger than Peter. Is Ann younger than John? Yes. You need a puzzle solving program, dudd. $\endgroup$ Feb 17, 2018 at 10:41
  • $\begingroup$ Have you solved this problem yet? $\endgroup$ Jan 4 at 16:34

1 Answer 1


This problem is very challenging since you need to evaluate the quality of the candidate answers.

In question answering, there are common steps that you need to follow. To summarise you need first to find the sentence that can answer the question and then compose the final answer.

In the first step you can measure the semantical similarity between Q and A, this a first filter where you can use several deep learning methods. Also, you can define a threshold to validate if the pair QA are enough related.

In the second step, you must extract the answer, if the answer is a fact you can use KB extraction, or if it is a summary, list you can use other DL methods. There is also the possibility that you must infer the answer, also suitable for DL methods.

I suggest to check this paper:

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .