I've been researching AI regulation and compliance (see my related question on law.stackexchange), and one of the big take-aways that I had is that the regulations that apply to a human will apply to an AI agent in most if not all cases. This has some interesting implications when you take a look at concepts like bias and discrimination.
In the case of a model with explicit rules like a decision tree or even a random forest, I can see how inspecting the rules themselves should reveal discrimination. What I'm struggling with is how do you detect bias in models like neural networks, where you provide the general structure of the model and a set of training data, and then the model self-optimizes to provide the best possible results based on the training data. In this case, the model could find biases in past human decisions that it was trained based on and replicate them, or it could find a correlation that isn't apparent to a human and inform decisions based on this correlation that may result in discrimination based on a wide array of factors.
With that in mind, my questions are:
- What tools or methodologies are available for assessing the presence and source of bias in machine learning models?
- Once discrimination has been identified, are there any techniques to eliminate bias from the model?
