0
$\begingroup$

The thing about machine learning (ML) that worries me is that "knowledge" acquired in ML is hidden: we usually can't explain the criteria or methods used by the machine to provide an answer when we ask it a question.

Edit: It's as if we asked an expert financial analyst for advice and he/she replied, "Invest in X"; then when we asked "Why?", the analyst answered, "Because I have a feeling that's the right thing for you to do." It makes us dependent on the analyst. end edit

Surely there are some researchers trying to find ways for ML systems to encapsulate and refine their "knowledge" into a form that can then be taught to a human or encoded into a much simpler machine. Who, if any, are working on that?

$\endgroup$
  • $\begingroup$ quote “Knowledge representation is a topic poorly discussed in machine learning.” Clark, Peter. "Knowledge representation in machine learning." Machine and Human Learning (1989): 35-49. In most working systems it's realized with features. For example one feature for the position of the pong ball, another feature for the pong paddle and so on. $\endgroup$ – Manuel Rodriguez Nov 1 at 14:42
  • $\begingroup$ 1989 was a long time ago! Hopefully some researchers have taken up the challenge and made useful progress. $\endgroup$ – S. McGrew Nov 1 at 15:12
0
$\begingroup$

I have one concept, which allows to structure knowledge gathered by a ML system in both categorical and algorithms-like structure.

The key idea here is that we contain in our minds some network-like structure in our minds, which helping us not only to classify some data, but also to form an abstract meaning about text or message.

Other than that, in opposite to our minds, it looks like this solution can produce these structures in readable for us forms.

$\endgroup$
0
$\begingroup$

Good question. We actually can explain how the machine answers our questions. For example, we know how the ML works for the case of image recognition. See the following paper by Chris Olah:

https://distill.pub/2017/feature-visualization/ and also https://distill.pub/2018/building-blocks/

By feature visualization, we know that the machine sees at each layer and how it decides using given information. As the images are visual it is easy to "visualize" the features but of course, interpreting inner workings of a Neural Network for other kinds of data (such as financial data, for example) would be much more difficult I presume. But still, we can understand how the initial data is weighted or how the intermediate features are calculated.

Given that, I highly doubt the reasoning of a machine can be taught to a human. It is far too complex to be learned. We can easily understand one or two dimensional functions, but the data we use in machine learning problems usually have hundreds of dimensions.

$\endgroup$
  • $\begingroup$ When human engineers design a system, say a moon rocket, of course computers are used to speed the calculations, but the engineers could "do it by hand". A moon rocket can contain hundreds of thousands of components. Because the design is hierarchical, it is possible to train an engineer to do any "local" or "global" part of the design process, and understand what he's doing and why it it necessary. A human being could be trained to do any small enough part of what a machine learning system does, but unless the ML system is hierarchical, it's hard for humans to understand the "why". $\endgroup$ – S. McGrew Nov 2 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.