1
$\begingroup$

I'm trying to process product data for an e-commerce platform. The goal is to understand products' size.

Just to show you some examples on how messy product dimension description is:

Overall Dimensions: 66 in W x 41 in D x 36 in H
Overall: 59 in W x 28.75 in D x 30.75 in H
92w 37d 32h",
86.6 in W x 33.9 in D x 24 in H
W: 95.75\" D: 36.5\" H: 28.75\"",
W: 96\" D: 39.25\" H: 32\"",
"118\"W x 35\"D x 33\"T.",
"28 L x 95 W x 41 H"
"95\" W x 26.5\" H x 34.75\" D"
"98\"W x 39\"D x 29\"H"
"28\" High x 80\" Wide x 32\" Deep"

Now assume that the product dimension description is short < 60 characters, I trained a two layer bidirectional LSTM, which can handle this task perfectly.

But the problem is, the above dimension is usually embedded in a long context (as a part of the product description). How can I extract the useful information from the long context and understand it? My LSTM can only accept context size of 60.

What language model is more suitable for this?

$\endgroup$
2
  • $\begingroup$ Why can you only train an LSTM with a maximum context of 60? $\endgroup$ May 11, 2020 at 17:17
  • $\begingroup$ I would also use (in addition of LSTM) some symbolic AI approach, expert-system like (e.g. RefPerSys....) $\endgroup$ Dec 5, 2021 at 20:53

1 Answer 1

1
$\begingroup$

In this problem, you can also use NER models in order to tag those numbers as Width, Height,... . You can also fine-tune a DistilBert model for your task.

if you want to train NER model by tok2vec you can use: Spacy Library

for fine-tuning DistilBERT you can use this. Hugging Face

$\endgroup$
1
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Dec 4, 2021 at 15:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .