First and foremost, I have to say that this could (and likely will) be a very hard task. Neural networks (NNs) have excelled at computer vision tasks identifying everything from textures to complex objects but what you are trying to do goes beyond that. We (humans) identify trash using the context as much as the object. An object on a table and the same object in a trash bin can look identical but the context tells us which one is trash. Also, anything can be trash. Trash is not an object, its a state.
Having said all that, it sounds like an interesting project and it would be a very useful model so I'll do my best to explain how you would go about trying this. As per your comment, you have outlined the two steps you need.
- Identify and extract the trash object from the image.
- Classify the type of waste
This could be accomplished using a single model but I'll explain how to do it with two models to make clear what is being done.
Identify and extract trash
The objective of the first model is to identify and extract the region of interest (the part of the image containing the trash). This is an image segmentation task typically accomplished using an R-CNN - some guides explaining how they work can be found here, here or here. These methods use supervised learning which requires masks delineating the object of interest to be used as the ground truth. A mask is a binary image the same size as your input image where each pixel in the image representing your positive class (trash) is set to 1 and all others are set to 0.
The output of your Canny/Watershed algorithms could be used as the masks to train the model, however your model will only ever be as good as your masks. Therefore, you might as well use your Canny/Watershed algorithms to accomplish task 1. If the masks generated by your algorithm are not of sufficient accuracy you will need to find another way to generate your masks - maybe even doing it manually.
An approximate rule of thumb for object detection with NNs is that you need 1,000 representative images per class, where class implies a specific object. In this case, unless you are attempting to identify very specific items of trash, you would likely need 100s of thousands of images to obtain a high degree of accuracy.
Classify the type of waste
This task should be easier, making the assumption that the segmented image from the first step is always trash. By assigning a label to each segmented image (paper, metal, glass, cardboard, etc) it becomes a normal multi-class classification task which are well documented online with lots of explainations and tutorials.
Single model
These tasks could be combined into a single model by modifying the masks in step 1. Instead of using a single binary mask, you could create a multichannel mask of shape m x n x o where m x n is your input image size and o is the number of kinds of waste you are attempting to identify. Each channel is a binary mask for a given type of waste. Therefore, it becomes a matter of not just identifying and segmenting trash but of segmenting each type of trash separately. Needless to say the complexity of the model would be a lot higher and so would the process to create the masks.
objectfrom the image of garbage and then predict what kind of waste it is (paper,metal,glass,cardboard etc) using a Keras model. There is just one object per image. $\endgroup$ – Deshwal May 5 '20 at 7:09