My first recommendation would be before you create an AI or ML based solution. Kindly consider using a business Q&A Software such as Questions for Confluence by Atlassian among others. An enterprise multiple choice solution could be a simple and elegant fit for this problem.
However, if one was to design a solution in accordance to your specification the first step to resolve the problem would definitely be one of applying NLP. In my answer I utilize the open source procedure laid out by EASE (Enhanced AI Scoring Engine) by Edx and a combined initiative of MIT and Harvard towards improving automated essay scoring (Parachuri V 2013).
Computers cannot directly understand text the way people can. People naturally break sentences into units of meaning, however when using a computer we will first have to instruct the computer how to do it. This process is called tokenization. Tokenization is the task of chopping a phrase into pieces called tokens at the same time discarding some characters i.e. punctuation.
After we tokenize the essay answers, we then convert the tokens into a matrix (bag of words model also known as vector space model). This model is a very convenient way to represent text in matrix form. Here we disregard grammar and even the word order but the frequency of words is retained.
Now we can proceed to the machine learning part!
Here I will recommend that we either use regression or classification. The difference between these two algorithms is that regression assumes you are using a continuous scale while classification is discrete.
In statistical modeling, regression is a set of statistical processes for estimating the relationships in variables.
A simple linear equation is $y=m*x+b$ , where y is the target value(score), m is a coefficient, and b is a constant.
In linear regression, we would do something like $y=m{1}*x{1}+m{2}*x{2}+\dots+m{n}*x{n}+b$.
Classification (also sometimes called a decision tree) on the other hand is a data mining method for assigning categories to a collection of data which is useful since we want to classify essays on the basis of their scores.
The next step will be to measure the error in the scores that are generated by the algorithm on the previous step.
In order to measure accuracy, lets use cross-validation. Here we split the training dataset randomly into n parts each segment is a 'fold'. This is called n-fold cross validation. We then iterate from 1 to n predicting the scores of the parts [n] from all the data in parts [!n].
Hope this helps.
