I have a model that predicts sentiment of tweets. Are there any standard procedures to evaluate such a model in terms of its output?
I could sample the output, work out which are correctly predicted by hand, and count true and false positives and negatives but is there a better way?
I know about test and training sets and metrics like AUROC and AUPRC which evaluate the model based on known data, but I am interested in the step afterwards when we don't know the actual values we are predicting. I could use the same metrics, I suppose, but everything would need to be done by hand.