I'm trying to do some research about semantic segmentation for webpages, in particular e-commerce webpages. I found some articles which provide some solutions based on very old dataset and those solutions in my opinion can't be effective for modern websites, in particular e-commerce. I would like to semantically infer the images bounding box, text, price etc..

Another problem is related with the size of webpage screenshot which are huge, I resized to 1024x512, but I think that I can't resize the image more otherwise I loose quality.

I built a very complex neural network in order to semantically infer text, images and background, (not classification but just segmentation), and the results are not so bad, but they are far from my expectations which seems strange to me, as we have many DNN able to do semantic segmentation of road, building, car etc for example. One problem is for sure the lack of a dataset with detailed labels. I didn't find any dataset that can satisfy my requests.

QUESTION: Any idea to help the network learn better the structure of a webpage just with a screenshot?

My DNN essentially is built as an auto-encoder architecture based on Segnet, with some modifications, skip connections, unpooling etc, I think that it is a good network.

references: https://clgiles.ist.psu.edu/pubs/CVPR2017-connets.pdf https://link.springer.com/chapter/10.1007/978-981-13-0020-2_33

  • $\begingroup$ You say "I found some articles which provide some solutions". You should maybe cite 1-2 of those articles (and maybe describe why those solutions don't work). Then you say " but they are far from my expectations". What are exactly your results and expectations. Please, focus on 1 question. Here you're asking many questions. $\endgroup$
    – nbro
    Commented Nov 4, 2020 at 11:10
  • $\begingroup$ I edited the question with some references. I though my expectations was clear, semantic segmentation of e-commerce webpages to identify where is text, images, maybe the price, but not necessary, it is important to identify images and text bounding boxes. Sorry for asking you different questions, I would like to create a nice discussion about this topic, covering all the possible things in just one post. $\endgroup$
    – FraMan
    Commented Nov 4, 2020 at 11:59
  • $\begingroup$ Our site is not appropriate for discussions, but for specific questions that lead to specific/non-subjective answers. So, I really suggest that you ask only one question. If you have multiple questions, ask each of them in its separate post. $\endgroup$
    – nbro
    Commented Nov 4, 2020 at 12:10
  • $\begingroup$ Ok, edited again $\endgroup$
    – FraMan
    Commented Nov 4, 2020 at 12:50


You must log in to answer this question.

Browse other questions tagged .