How to classify two very similar images using Deep Learning?

Question

I am a newbie in Computer Vision.

I have a scenario in which I have a stationary camera in a factory. I want to detect whether the technician is working on the machine or not.

Images are like the following:

Technician working:

Technician absent:

Technician not working:

I am confused whether is it a Image classification issue or an Object Detection/Pose Detection problem.

As per my knowledge this should be a classification problem, I should take multiple images of a condition in which the machine is unattended, and a condition in which the technician is working on the machine.

I will train the model if with different individual technicians on different days with different clothes.

Now if I am in the right direction, how much images do I need to have a good accuracy?

I see there are different models on Tensorflow Hub on image classification like EfficientNet, etc. Which model/architecture will work for me?

I am sorry if I sound noobish.

I can train the model using simple classifiers' code (like Cat vs Dog), but I want the my architecture to understand that there is an area in the image which should only be checked if it is occupied or not to classify properly.

OR

Shall I cut out the middle area (where technician stands) simply using opencv. And then feed that cutout image to some classifier to detect if there is a human standing there?

Thanks in advance!

Answer 1

This is a bit old question, but I'll answer anyway. Naturally the more you have data the better, but rather than capturing an image (for example) every second, I would rather capture an image every 5 minutes for a period of at least 30 days. This would give you 8 * 60 / 5 = 96 pictures per work day. But naturally you can start developing the algorithm at day one.

Generating test & validation sets on "time series" data can be tricky, I would assign complete work days to the validation set rather than taking random samples. This way you can test that the code works even when people are wearing different clothing, and the weather could be different as well.

For supervised learning you must manyally classify all of the images, since there aren't that many images I would go this router rather using a more difficult semisupervised learning. As a human you have some domain expertice on the image content, I would crop the image quite tightly around the area of interest so the network doesn't need to learn to ignore certain areas by itself.

You could have some edge-cases to classify, for example if a technician is just standing in front of the machine, is he considered working? How about if the is facing the other way? Depending on how you want to handle these, your network can either be very simple or a bit more advanced.

Maybe you don't need a neural network at all, and you can just pass a low-res image through LDA / Fisher Linear Analysis. Or create pixel-wise histograms of the colors for working and non-working image classes separately, and create heuristics based on that.

Once you have images from different days, pay attention on what kind of variance there is in colors, brightness, camera noise etc. Then create a data augmentation pipeline which mimics the observed deviations, this way you can make the model much more robust and you don't need to wait several months to start getting good results.

Answer 2

What you are asking about can be treated as a classification problem, indeed. But I would treat it rather as a detection problem.

Given an image, the goal is to draw the bounding box or any other geometric shape around the object of interest. In other words, you would like to localize and identify the object.

I recommend reading this link https://machinelearningmastery.com/object-recognition-with-deep-learning/.

Good repository for detection - https://github.com/ultralytics/yolov5.