I am pretty new to AI and have recently been paying attention to AI explainability and the fact that it remains a hurdle within the path of commercializing certain AI systems in health for instance. I tried to do some digging myself by starting with the gradient descent algorithm as an optimization technique used to model error of predictions. Ideally, I am aware it is most suitable for predicted errors that fit a Gaussian distribution. One question I have not been able to find answers to and would really appreciate help from this community
What exactly does the AI explainability problem refer to? Is it related to the convex optimisation (G.D.) process ? Or randomly generated coefficients? How exactly do we mean when we say AI is a blackbox?
p.s: If this has already been answered elsewhere on this platform do share a link.
Further clarification on my question:
Please, reference this tweet by Y. Lecun only yesterday, I would like some more responses on relationship between the AI explainability problem and the weight optimization technique. How or why is G.D applied to optimising parameter weights contribute to AI being a blackbox. Or if this is another separate problem from AI explainability also do clarify.