Questions tagged [explainable-ai]

For questions related to explainable artificial intelligence (XAI), also known as interpretable AI, which refers to AI techniques that can be trusted and easily understood by humans, which are particularly relevant in areas like healthcare or self-driving cars. There are several concepts related to XAI, such as accountability, fairness, and transparency.

9 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2
votes
0answers
38 views

What is the paper that states that humans incorrectly trust the incorrect explanations of the AI?

I was reading a paper on the subject of explainable AI and interpretability, in particular the tendency of people (even experts) to excessively trusting explanations given by AI. In the intro the ...
2
votes
1answer
131 views

Black Box Explanations: Using LIME and SHAP in python

Recently, I came across the paper Robust and Stable Black Box Explanations, which discusses a nice framework for global model-agnostic explanations. I was thinking to recreate the experiments ...
1
vote
0answers
50 views

What exactly do gradient-based saliency map tell us?

As far as I understand, gradients are supposed to tell us 1) the magnitude and 2) direction, to update a parameter such as to minimize the loss function. Regarding saliency maps, which use gradients ...
1
vote
0answers
15 views

What does the lambda parameter in the paper "Interpretable Explanations of Black Boxes by Meaningful Perturbation" do?

I do not understand the purpose of the $\lambda$ parameter in equation 3 of the paper Interpretable Explanations of Black Boxes by Meaningful Perturbation. $$m^{*}=\underset{m \in[0,1]^{\Lambda}}{\...
1
vote
0answers
57 views

Is it possible to create a fair machine learning system?

I started thinking about the fairness of machine learning models recently. Wiki page for Fairness_(machine_learning) defines fairness as: In machine learning, a given algorithm is said to be fair, ...
1
vote
0answers
54 views

What should we do when the new data drastically change the current model?

In machine learning (in particular, supervised learning), if some new data changes the previous model/function drastically, then I think we should study that data. Does it happen? How to handle such a ...
0
votes
0answers
27 views

Explainable AI for complex input features

I have a model for binary classification that includes 2 linear layers with RELU activation function and Sigmoid in the last layer. The input features are FastText word embedding, frequency, and ...
0
votes
0answers
15 views

Why can we compute mutual information in deep neural networks in information bottleneck context?

In the famous Information bottleneck paper by Tishby(https://arxiv.org/abs/1703.00810), the author proposed a framework that the neural network can compress information. And they computed the mutual ...
0
votes
1answer
66 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...