1
$\begingroup$

I'm trying to create a distilled model of SAM https://colab.research.google.com/drive/1PKc5x1XjxOPP-2msFp6uMSULvnm9dorw?usp=sharing

I'm able to label the data and train, but the predict doesn't find anything

Please help

I tried:

  • remove the whole doc BB by removing the biggest bounding box, it still return empty
  • Train on JPG/PNG, same result

NOTE that I am able to train with autodistill/GroundedSAM on the same dataset (I use it to find the document boundaries for auto cropping)

The purpose is to find bounded area in a document

SAM can do it well but it's heavy and slow, therefore I'm trying to create a Autodistill YOLOv8 that will find the bounded area in a document

I'm not sure that YOLOv8, because the concept of document is foreign to YOLOv8, is there a better way to create a SAM distilled model?

I can see that the labels are assign correct: enter image description here

$\endgroup$

1 Answer 1

1
$\begingroup$

It looks like the problem is that it is also predicting a bounding box around the whole document. Overlapping boxes of the same class will confuse the model training process (I believe only the biggest one will get used).

You may want to try some different prompts in your ontology to see if you can get it to label things better.

$\endgroup$
6
  • $\begingroup$ I change the code to remove the whole doc BB by removing the biggest bounding box, it still return empty $\endgroup$
    – Elia Weiss
    Commented Jun 12, 2023 at 8:51
  • $\begingroup$ also, it should have return the whole doc BB, not empty $\endgroup$
    – Elia Weiss
    Commented Jun 12, 2023 at 8:51
  • 1
    $\begingroup$ Looking at the training logs in the Colab it looks like not nearly enough images in the training set (8; try with at least 100) and for such a small dataset you will want to train for many more epochs (it only got to 2% mAP after 50). $\endgroup$
    – Brad Dwyer
    Commented Jun 13, 2023 at 12:29
  • $\begingroup$ I tried with more images (87), I would expect it to at least return something, note that I use the same dataset to train another model with autodistill and GroundedSAM and it was working ok $\endgroup$
    – Elia Weiss
    Commented Jun 13, 2023 at 12:57
  • $\begingroup$ Is this all in the same Colab environment? Maybe try resetting it and starting from scratch $\endgroup$
    – Brad Dwyer
    Commented Jun 15, 2023 at 12:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .