# In this paper, if region $R_{2}$ moves in a sliding window manner, won't the saliency map have a smaller size than the original image?

In the paper Salient Region Detection and Segmentation, I have a question pertaining to section 3 on the convolution-like operation being performed. I had already asked a few questions about the paper previously, for which I received an answer here. In the answer, the author (JVGD) mentions the following:

So for each pixel in the image you overlap on top $$R_{1}$$ and then $$R_{2}$$ on top of $$R_{1}$$, then you compute the distance $$D$$ for those 2 regions to get the saliency value of that pixel, then slide the $$R_{1}$$ and $$R_{2}$$ regions in a sliding window manner (which is basically telling you to implement it with convolution operation).

Regarding the above, I had the following question: If the region $$R_{2}$$ moves in a sliding window manner, won't the saliency map (mentioned in section 3.1) have a smaller size than the original image (like in convolution the output image is smaller)? If this is so, wouldn't it be impossible to add the saliency maps at different scales since they each have different sizes?

The following edit re-explains the question in more detail:

In the animation above, you can see a filter running across an image. For each instance of the filter, some calculations are happening between the pixels of the filter and the image. The result of each calculation becomes one pixel in the output image (denoted by "convolved feature"). Here, the output image is smaller than the input image because there are only 9 instances of the filter. From what I understood of the salient region operation, a similar process is being followed i.e. a filter runs across an image, some calculations happen, and the result of each calculation becomes one pixel in the output image (saliency map). Hence, won't the saliency map have a smaller size than the original image? Furthermore, when the filter size is 3 x 3, the output image size is 3 x 3. However, if the filter size was 5 x 3, the output image size would only be 1 x 3. Clearly, the output image size is different for different filter sizes. This makes the output images (saliency maps) impossible to add. There is clearly something I am missing / misunderstanding here, and clarity on the same would be much appreciated.

P.S. There is no indication of padding or any operation of that sort in the research paper, so I don’t want to assume anything because the calculations would then be wrong.

• Have you considered they just pad the input? Oct 15, 2021 at 9:40

The paper you have provided, discusses *detecting salient regions by using a contrast determination filter

which operates at various scales to generate saliency maps containing “saliency values” per pixel.*

The Contrast detection filter used in this paper is better explained with the figure 2. Here R1 is the inner most region and R2 scale is varied. Note this is the filter and not the image itself

The below fig 3 shown in the paper section 3.1 indicates how the image appears when the filter of varying scales is applied(filtered images). When R2 has the highest scale(i.e., maximum width) the background is also shown that is the non-salient parts are also taken into consideration. As the R2 takes up lesser width, the non-salient parts becomes almost invisible which you can see in 3 to 8 images in figure3. It helps to focus on what's really important in the image, which is the man riding the horse.

If you have noted in the figure 3, though R2 width varies, the filtered image size is the same as the original image.
To answer your question, the saliency map is the same size as the original image size as only the filter was scaled not the image itself

This is also quoted in the paper section 3.1, page 4

A change in scale is affected by scaling the region R2 instead of scaling the image.
Scaling the filter instead of the image allows the generation of saliency maps of the same size and resolution as the input image.

Update Let me provide you with a much more simpler explanation ,
suppose you as a human have couple of reading glasses(of varying sizes) and you are trying to read a book.

• The smallest reading glass will help you focus on smaller part of the book, and you would often have to look in different parts of the book
• A medium reading glass will focus on a more wider part of the book
• If you have a large reading glass, you can cover the whole book

Now in above example, reading glass acts as the filter, the book acts as your input image, the output image is what you can see through the reading glass.

No matter the size of your reading glass(filter) the dimensions of the book hasn’t changed only the focus on what is observed i.e., content or the information of book you see through your reading glass is more focused and everything else is a blur.

So the output image will be the same size as the input image with the out of focused will be like a blur

• @ArchanaDavid I don't think this really answers my question. I have edited my original question to provide more detail and clarity for what the question actually is. Hopefully, you can provide more insight on how exactly "scaling the filter instead of the image allows the generation of saliency maps of the same size and resolution as the input image".
– user48670
Oct 19, 2021 at 13:05
• @learning123 have updated my answer to help you better understand the concept Oct 19, 2021 at 13:32
• @ArchanaDavid I am trying to code the algorithm in the research paper for a project, hence I would highly appreciate it if you could explain the process exactly (maybe with a little diagram or something or something like how I explained my logic with the above animation) or tell me what is the flaw in my reasoning because although I understand your analogy, it doesn't explain how the output image sizes won't be different for different filters (you can refer to my question for my logic). Sorry if I am stretching this, but I am still confused about what is exactly happening with the filter.
– user48670
Oct 19, 2021 at 13:54
• @ArchanaDavid I think I am having some confusion with the filtering process itself in thinking that it is like a convolution operation and that is why I am having issues. If you can explain that process in a little bit if detail, I think it will answer my question. Once again, sorry for constantly pestering you.
– user48670
Oct 19, 2021 at 14:03
• "operates at various scales...pixel" appears to have been copy-pasted from the paper. Please see our rules on copying from external sources: ai.stackexchange.com/help/referencing. In particular, whenever you copy from another source, we require you to clearly indicate that you are quoting from them and credit the source.
– D.W.
Dec 27, 2021 at 2:21