2
$\begingroup$

The thing about machine learning (ML) that worries me is that "knowledge" acquired in ML is hidden: we usually can't explain the criteria or methods used by the machine to provide an answer when we ask it a question.

It's as if we asked an expert financial analyst for advice and he/she replied, "Invest in X"; then when we asked "Why?", the analyst answered, "Because I have a feeling that's the right thing for you to do." It makes us dependent on the analyst.

Surely there are some researchers trying to find ways for ML systems to encapsulate and refine their "knowledge" into a form that can then be taught to a human or encoded into a much simpler machine. Who, if any, are working on that?

Improve this question
$\endgroup$
1
  • $\begingroup$ Please, next time when you ask a question, put your question also on the title. I think that this helps to attract more people because they can have an idea of what your post is about. Btw, you're looking for "explainable artificial intelligence". $\endgroup$
    – nbro
    Jul 29, 2020 at 21:42

3 Answers 3

Reset to default
0
$\begingroup$

I have one concept, which allows to structure knowledge gathered by a ML system in both categorical and algorithms-like structure.

The key idea here is that we contain in our minds some network-like structure in our minds, which helping us not only to classify some data, but also to form an abstract meaning about text or message.

Other than that, in opposite to our minds, it looks like this solution can produce these structures in readable for us forms.

Improve this answer
$\endgroup$
0
$\begingroup$

I think you are looking for the field known as explainable artificial intelligence. The book Interpretable Machine Learning: A Guide for Making Black Box Models Explainable will surely help you to understand the issues and existing techniques. See also the following question Which explainable artificial intelligence techniques are there?.

Improve this answer
$\endgroup$
0
0
$\begingroup$

We actually can explain how the machine answers our questions. For example, we know how the ML works for the case of image recognition. See the following paper by Chris Olah:

https://distill.pub/2017/feature-visualization/ and also https://distill.pub/2018/building-blocks/

By feature visualization, we know that the machine sees at each layer and how it decides using given information. As the images are visual it is easy to "visualize" the features but of course, interpreting inner workings of a Neural Network for other kinds of data (such as financial data, for example) would be much more difficult I presume. But still, we can understand how the initial data is weighted or how the intermediate features are calculated.

Given that, I highly doubt the reasoning of a machine can be taught to a human. It is far too complex to be learned. We can easily understand one or two dimensional functions, but the data we use in machine learning problems usually have hundreds of dimensions.

Improve this answer
$\endgroup$
1
  • $\begingroup$ When human engineers design a system, say a moon rocket, of course computers are used to speed the calculations, but the engineers could "do it by hand". A moon rocket can contain hundreds of thousands of components. Because the design is hierarchical, it is possible to train an engineer to do any "local" or "global" part of the design process, and understand what he's doing and why it it necessary. A human being could be trained to do any small enough part of what a machine learning system does, but unless the ML system is hierarchical, it's hard for humans to understand the "why". $\endgroup$
    – S. McGrew
    Nov 2, 2019 at 14:36

Not the answer you're looking for? Browse other questions tagged .