9
$\begingroup$

I'm starting a project that will involve computer vision, visual question answering, and explainability. I am currently choosing what type of algorithm to use for my classifier - a neural network or a decision tree.

It would seem to me that, because I want my system to include explainability, a decision tree would be the best choice. Decision trees are interpretable, whereas neural nets are like a black box.

The other differences I'm aware of are: decision trees are faster, neural networks are more accurate, and neural networks are better at modelling nonlinearity.

In all of the research I've done on computer vision and visual question answering, everyone uses neural networks, and no one seems to be using decision trees. Why? Is it for accuracy? I think a decision tree would be better because it is fast and interpretable, but if no one's using them for visual question answering, they must have a disadvantage that I haven't noticed.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
8
$\begingroup$

For vision tasks, neural network models almost always include a number of layers that pool and convolute. The convolutions, in particular, are very useful - they can make the model generalize better to inputs and maintain performance when inputs have undergone certain linear transformations (e.g. some scaling or a translation along the x-axis). These properties, along with the robust frameworks that exist for developing and deploying neural nets, and the fact that they have been shown to widely produce very good results, are some of the reasons they’re used.

In terms of being a black box, while this is true for a lot of applications it’s actually less true for image-based tasks. The layers of a well-designed and trained convolutional neural network model can actually be visualized and made quite interpretable; from these visualizations, it’s often clear how the representation roughly works. In contrast, I’d argue that while a decision tree is theoretically easier to interpret for some tasks (say medical decision making), this is less the case for vision tasks because we don’t interpret images one pixel at a time or using some readily available feature of the image, such as the width, height, or color frequencies. People are almost always interested in the higher-level representation within an image (say, a cat, a leaf, or a face), and that sort of feature extraction is exactly what CNNs are good at. Decision trees, in contrast, tend to have trouble capturing these higher-level representations.

Distill.pub has a nice explanation of feature visualization that may be of interest.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thanks Greenstick! I read about CNNs a while ago but had completely forgotten - now that you've explained I can't believe I didn't think of it right away haha. So if I understand correctly, a CNN would be more suitable than a decision tree for computer vision, because CNNs can represent features the same way regardless of location within the image, whereas decision trees can only represent a 'group of pixels' rather than the higher-level idea of features because the lack the convolution and pooling layers. $\endgroup$
    – The Impossible Squish
    Apr 27, 2018 at 2:51
  • 3
    $\begingroup$ Yes, more or less, and it’s not just translation invariance but also size (scale) and rotation. Decision trees can represent more than just a group of pixels; I would be remiss to not say that feature engineering is often 90% of the work. A decision tree model can benefit hugely from that but interpretability will take a dip, depending on how you engineer your features. Generally, CNNs don’t require the feature engineering in a meaningful way because they were developed specifically for vision tasks. You can think of them as an algorithm specialized more for vision problems. $\endgroup$
    – Greenstick
    Apr 27, 2018 at 16:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .