4
$\begingroup$

I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised way, the positions of that object. For example, us, as humans, would record the x and y location of the object. Of course the NN doesn't have a notion of x and y, but i would like, given an image, the NN to produce 2 numbers, that preserve as much as possible from the actual relative position of objects on the background. For example, if 3 objects are equally spaced on a straight line (in 3 of the images), I would like the 2 numbers produced by the NN for each of the 3 images to preserve this ordering, even if they won't form a straight line. They can form a weird curve, but as long as the order is correct that can be topologically transformed to the right, straight line. Can someone suggest me any paper/architecture that did something similar? Thank you!

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ By fixed, you mean the background is always the same image ? $\endgroup$
    – Astariul
    Commented Sep 17, 2019 at 4:35
  • $\begingroup$ @Astariul yes! and the object that changes position is also the same in each image (same size, shape, orientation etc.). $\endgroup$
    – Silviu-Marian Udrescu
    Commented Sep 17, 2019 at 5:45
  • 1
    $\begingroup$ @Silviu-MarianUdrescu you mightn't need machine learning for this, it sound's like the object is very defined. If you can code something up that works 100% of the time why not do that? $\endgroup$
    – Recessive
    Commented Sep 17, 2019 at 6:09
  • $\begingroup$ @Recessive Ideally I want a NN that is able to learn the 2 numbers representation of an image, for any set of background and object moving. Coding it by hand works for only a fix background and a fix image (which is indeed what my post is about), but I want a NN approach so I can later generalize i.e. if I pass a new set of images (all with the same background and object, but different from the ones in the previous set of images) the NN would identify the "x" and "y" just as easily without any coding modifications. Coding something up manually would require a new code for each set of images. $\endgroup$
    – Silviu-Marian Udrescu
    Commented Sep 17, 2019 at 6:15
  • $\begingroup$ @Silviu-MarianUdrescu There should be a few ways of doing this. You could create a regression CNN that outputs (x,y) coordinates (i wouldn't recommend this, it's very hard to get this to work from my experience). You could use deconvolutional layers to produce an output image of similar dimensions to the input image, using a softmax on the entire image to produce a probability of the image being at each location (you could also produce a downscaled version if approximate location is ok). Other then that I think there's so great resources online for object detection, just google search. $\endgroup$
    – Recessive
    Commented Sep 17, 2019 at 6:29

1 Answer 1

Reset to default
1
$\begingroup$

As said in the comments, I wouldn't use Machine Learning for that.

You can achieve that result using something like OpenCV.

For example:

  1. Get the "Naked" Background image: If you don't have it, you can easily calculate it by making an average of each image: background = np.mean(images, axis=0)
  2. For each image, calculate the pixel difference between image and background. diffs = [img - background for img in images]
  3. Diff's pixels can be negative, so take the absolute value of each pixel before converting it to grayscale.
  4. If all goes well, you now have a dark noised image, with a bright silhouette of your object.
  5. Set a threshold (i.e. threshold = diff.percentile(95)) and make a binary mask, so now each pixel indicates 1 for image silhouette and 0 for background.
  6. Find the centroid of the object (like calculating the average coordinates for each pixel=1). And there you have it!

Of course, I just described one clear and easy way to do it. But you can find your own best solution.

  • ✅ Don't need to train a neural network
  • ✅ Don't need to label data
  • ✅ Works for any set of image / background
  • ✅ Precise coordinates
  • ✅ Easy to make, debug and adapt.
  • ✅ Runs fast
Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .