# How to use 'Canny/Watershed' algorithm's output as an input for Image Classification Model

I have a very silly problem in hand. I have implemented 2 methods which give me the mask to separate the objects from the background. What I get from one method is the object encapsulated in the red Contour or boundary and the other one makes the background Red.

I am using Keras to classify Trash. I wanted to use the output or the EXTRACTED object as an input to the CNN model. Now I do not see any difference in the images and output. All there is extra boundary around the object and I fail to understand how it can help my model.

I could add an extra alpha channel to second method to make the background transparent but Keras' ImageDataGenerator do not work with RGBA images. What shoul dI do to improve the model?

• By 'classify trash', do you mean a binary classifier which gives a boolean output for a given image (trash in image: true/false), or are you attempting to segment the trash in the image (a binary mask indicating which pixels represent trash in the image) – tynowell May 4 '20 at 21:03
• I want to extract the object from the image of garbage and then predict what kind of waste it is (paper,metal,glass,cardboard etc) using a Keras model. There is just one object per image. – Deshwal May 5 '20 at 7:09

First and foremost, I have to say that this could (and likely will) be a very hard task. Neural networks (NNs) have excelled at computer vision tasks identifying everything from textures to complex objects but what you are trying to do goes beyond that. We (humans) identify trash using the context as much as the object. An object on a table and the same object in a trash bin can look identical but the context tells us which one is trash. Also, anything can be trash. Trash is not an object, its a state.

Having said all that, it sounds like an interesting project and it would be a very useful model so I'll do my best to explain how you would go about trying this. As per your comment, you have outlined the two steps you need.

1. Identify and extract the trash object from the image.
2. Classify the type of waste

This could be accomplished using a single model but I'll explain how to do it with two models to make clear what is being done.

## Identify and extract trash

The objective of the first model is to identify and extract the region of interest (the part of the image containing the trash). This is an image segmentation task typically accomplished using an R-CNN - some guides explaining how they work can be found here, here or here. These methods use supervised learning which requires masks delineating the object of interest to be used as the ground truth. A mask is a binary image the same size as your input image where each pixel in the image representing your positive class (trash) is set to 1 and all others are set to 0.

These tasks could be combined into a single model by modifying the masks in step 1. Instead of using a single binary mask, you could create a multichannel mask of shape m x n x o where m x n is your input image size and o is the number of kinds of waste you are attempting to identify. Each channel is a binary mask for a given type of waste. Therefore, it becomes a matter of not just identifying and segmenting trash but of segmenting each type of trash separately. Needless to say the complexity of the model would be a lot higher and so would the process to create the masks.