Are Computer Vision and Digital Image Processing part of Artificial Intelligence?

Question

There are some fields of Computer Vision that are similar to Artificial Intelligence. For example, pattern recognition and path tracking. Based on these similarities, can we say that the Computer Vision is a part of Artificial Intelligence?

Answer 1

Artifical Intelligence is a rather fuzzily-defined field. It includes lots of subject areas. Sometimes that's because they replicate some behaviour or mental ability of creatures used to solve problems, and sometimes it is because they were the focus of "classic" AI, and have remained classified as part of AI. These problems include navigation, searching, solving combinatorial or logic puzzles, complex systems control, communication. Many of the problems can include converting visual input to something more useful to the rest of the system.

Computer Vision by comparison appears more tightly-defined. However, it quickly becomes less so when you try to consider what the eventual outputs should be. To what extent does a computer vision system need to "understand" the visual inputs in order for other system components to make sense of them, and use them correctly? Here are a couple of examples to illustrate my point:

Image captioning

For instance, work in automated captioning effectively includes natural language models. The input to the system is an image. The computer vision half of that system does not output a class or direct measurement of something inside the image. Instead it outputs a vector encoding of the description of the whole image - something that the language model can decode into an English sentence.

The vision model and language model are trained together, so this encoding can be very loosely considered an internal "ideagram" that represents the contents of the image and allows the vision and language models to share representations.

Agents using vision systems for decision-making

Another interesting example is recent work on Deep Q Networks, used to give superhuman performance when playing some Atari games. This used a neural network architecture (CNN) already known to be good for image classification problems, and used it to directly connect the pixel output from a game to a system which learned optimal behaviour. This is more clearly work in AI - creating a learning agent that works across a general set of problems - but at its core is a CV system.

In this case, the input is four consecutive images (although there are variations which learn from the stream of individual frames), and the output is a vector which scores the value of taking one of the possible controller actions (left, right, up, down, fire etc). Variations on the system directly output which action would be preferred from the CNN.

---

You could take this further and require that a vision system produce some kind of explainable, transferable, shared world model for other systems to interact with. An embryonic version of that approach is effectively how the captioning system works. It is not at all clear whether such an imagined design is strictly in the field of CV or some other part of AI.

Whether such outputs are considered part of computer vision or more generally part of AI is less of a pure scientific classification issue than an opinionated or political one (e.g. due to funding either an AI research team or a CV one).

The similarities between current computer vision approaches and ongoing work in AI, such as heavy use of deep learning models, would suggest that CV is strongly related to AI. Intuitively I would say that CV is part of AI, perhaps a key specialism of it, roughly equivalent in scope to Natural Language Processing. However, the practical outcome of that is only important to people who have labelled themselves "AI researcher" or "CV engineer" etc.

I also think the same arguments apply to other studies of artificial perception, most notably speech recognition.