0
$\begingroup$

In the context of the Machine Learning model, is there any clear definition of reliability, resiliency, and robustness of a model? I saw some papers discuss different things (e.g. attacked model, fault model, noisy data, etc.) when they talk about these terms.

$\endgroup$

1 Answer 1

1
$\begingroup$

I found this guy's thesis to be very helpful in understanding it:

  • Robustness: The model can be trained even with noisy and corrupted data. Simply put, if we add small perturbations to the image, can it still recognize it correctly? This one is important in Adversarial attack, where an attacker purposely adds the pertubation to misguide the models.
  • Reliability: After training and when deployed in the real-world, the model should not break down under benign shifts of the distribution. This is straightforward to understand: how good the model is on a set that has different distribution than the training set, e.g., I train on photos but evaluate on paintings.
  • Resilience: The modeling procedure should work under model mis-specification, i.e. even when the modeling assumption breaks down, the model should find the best possible solution. For example, when the hardware breaks, can the model still perform as good?
$\endgroup$
1
  • $\begingroup$ I see, it's a great lead, thank you! $\endgroup$
    – malioboro
    Mar 5, 2023 at 15:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .