Recent algorithms for correcting mislabeled data using multilayer perceptrons

Question

I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al. Please share if you are aware of recent/current updates with a brief thoughts. Thanks in advance.

Answer 1

The most general solution today for the problem of finding label errors in datasets is called "confident learning" which works for all datasets and models, can be run time-efficiently in one line of code using cleanlab, and has substantial theory to prove that it works in realistic conditions on real-world datasets. This "confident learning" paper was a culminating result during my PhD at MIT and I am an author on the paper.

• The Confident Learning Paper: https://arxiv.org/abs/1911.00068
• Code: https://github.com/cleanlab/cleanlab
• Examples of label errors found with cleanlab: https://labelerrors.com

Find label issues in your dataset in 1 line of code

from cleanlab.classification import CleanLearning
from cleanlab.filter import find_label_issues

# Option 1 - works with sklearn-compatible models - just input the data and labels ツ
label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)

# Option 2 - works with ANY ML model - just input the model's predicted probabilities
ordered_label_issues = find_label_issues(
    labels=labels,
    pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
    return_indices_ranked_by='self_confidence',
)

Train a model as if the dataset did not have errors -- 3 lines of code

from sklearn.linear_model import LogisticRegression
from cleanlab.classification import CleanLearning

cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
cl.fit(train_data, labels)

# Estimate the predictions you would have gotten if you trained without mislabeled data.
predictions = cl.predict(test_data)

Documentation and runnable tutorials for cleanlab: https://docs.cleanlab.ai/

The above approaches find issues. To correct label issues in your dataset:

The above approaches, just find issues and train without them. To correct the labels or train a model on a corrected dataset (that still includes all the errors, but now corrected with the right label), there is a no-code tool for that called Cleanlab Studio (https://cleanlab.ai/studio) for which I am also an author.

Background if you're interested:

I spent half a decade working with Isaac Chuang (inventor of the quantum computer) to solve this problem in a way that works for every dataset and every model (and every future dataset and future model) during my PhD at MIT. I originally decided to solve this problem after discovering (while building MIT and Harvard's cheating detection system in 2013) that most real-world datasets have significant label errors and this is one of the biggest problems that companies and universities struggle with when training ML models.