I'm trying to set up a pipeline for my ML models to automatically re-train themselves whenever concept drift occurs to recalibrate to the new output distributions. However, I can't get ground-truth from my data without manual labeling, and I want an automated MLOps pipeline, so that no human-in-the-loop labeling is required.
I was noticing that using zero-shot learning using BART was actually giving pretty impressive results with my label set on sample data I want to classify on.
Suppose I were to use BART to manually label data, say, if it scored my label with a score of 0.95 or better, and then just used that labeled data to train my models.
Is this "cheating"? Am I trying to cut a corner here that can't be cut? Will this create a meaningful model for my text data, or just cause my trained model to try to become a crude version of the BART model that provided the labels in the first place, and therefore just be worse than strictly using the BART model?