Is there any existing attempt to create a deep learning model which extracts vector paths from bitmaps?

Question

I need an algorithm to trace simple bitmaps, which only contain paths with a given stroke width.

Is there any existing attempt to create a deep learning model which extracts vector paths from bitmaps?

It is obviously very easy to generate bitmaps from vector paths, so creating data for a machine learning algorithm is simple. The model could be trained by giving both the vector and bitmap representation. Once trained, it would be able to generate the vector paths from the given bitmap.

This seems simple, but I could not find any work on this particular task. So, I suppose this problem is not fitted for current deep learning architectures, why?

The goal is to trace this kind of image, which would be drawn by hand with a thick felt pen and scanned:

So, is there a deep learning architecture fitted for this problem?

I believe this question could help me understand what is possible to do with deep learning and what is not, and why. Tracing bitmaps is a perfect example of converting sparse data to a dense abstract representation; I have the intuition one can learn a lot from this problem.

Answer 1

If we seek proven working source code to plug into a GPLv2-licence compatible solution, we should at least consider autotrace. Its source code is open for review. It can be tested against the example images we have and, if it works fine, called by our GPLv2 software. We can even use the calling code in Inkscape's plug-in image tracing implementation as a good starting point for design and implementation of our calling program, whether it be C, C++, Java, Python, or ECMA (JS).

The trace algorithm in Adobe Illustrator is comparable but is not open source.

If we seek theory, there are several academic publications discussing some of the theory, the last being most aligned with machine learning ideology. I would not dismiss earlier work simply because it doesn't connect with the current machine learning idioms. Investigating what is fully implemented and successfully used by many follows a wise old business proverb: The bird in the hand is worth two in the bush.

• Potrace: a polygon-based tracing algorithm, Peter Selinger, 2003

• Vector Representation of Binary Images Containing Halftone Dots, Kei Kawamura, Hiroshi Watanabe, Hideyoshi Tominaga, 2004

• Testing AutoTrace: A Machine-learning Approach to Automated Tongue Contour Data Extraction, Jae-Hyun Sung, Jeff Berry, Marissa Cooper, Gustave Hahn-Powell, and Diana Archangeli, 2013

Many of the online drawing programs collect data. It would not be surprising if, behind the gracious give-away of online bandwidth, they are establishing a continuously improving data set for training a new breed of autotracers. None have published AI designs admitting as much, but they would not be legally obligated to do so because a single input example is indeterminable from the autotrace service that could resulting from the training.

Answer 2

any luck update in this. I fully understand And today, I can tell u that my research has led me here, and that bitmap tracing for simple lines is so so so trash still after decades, And I have been a graphic designer for bout 20 years now. So anyway.im also a low level- mid level developer u could say, And for so many years I have been thinking of different ways to help solve this problem. And I dont know

But can some1 please advise me if this method could work

Though it's incomplete I'm hoping I'm on the right path and someone could add to it

Would it be effective to

Train a model on a dataset of bitmaps and it's corresponding SVG that will contain the max amount of points And % similarity average to the the bitmap (This idea stems around the thought of training a diffuser or something)

Then

Train another model(or something) On the same bitmaps dataset Though the corresponding SVG in this data set will be simplified to the least amount of paths and still within the same % similarity average to the bitmap

So that should simplify the paths and still look like the original (Obviously not theoretically because I'm not actually sure)

So now when you run this against a bitmap u want to vectorize

I think maybe Tiling the bitmap Scan all the tiles Reproduce that tile bitmap and corresponding SVG overlay with the trained model that produces the most points

And then stich the tiles back together And apply a final simplify using the Model trained with the least points

OK so as muddled as that all is really.. I'm hoping someone can understand that. And either tell me to shut up.

Or any help to develop this idea further or,

If you need a visual representation of what I'm trying to say, in order to understand it I can draw up something and send it

But really this has been a life long goal..

I would honestly design the entire dataset Just to shape the model to spec.

Or am I just way off the mark here😆😆