In NLP there is BERT which can take a sentence and turn it into an embedding (vector representation) which in some ways encompasses the "meaning" or more precisely the context of the sentance. Other networks can then use transfer learning on top of BERT and make predictions or extend it in various ways. I'm wonder if there is something similar to this but for images, where you input an image and then get back an image of the same spatial size but where each pixel is an embedding describing what it is in some abstract feature space. Ie it gives me back an image with the same size but more channels than the original image. Then one would be able to use transfer learning on top of this embedding and be able to for example process the image in various ways with this information of what it depicts.

To clarify, I'm not looking for an ordinary image segmentation network that give me one of a few predetermined classes. I want an abstract embedding representation for each pixel instead so it can handle classes it hasn't specifically been trained to recognize. In other words it has learned them in a kind of unsupervised way or as a side effect of supervised learning.

Does anything like this exist out there? And if it does, does it have pretrained weights so one can do transfer learning on it?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.