1
$\begingroup$

I have made some research online but I have yet to find a tool where I can input text such as:

A bonobo ape climbing a tree

And receive a image illustrating the sentence.

An API would be interesting too.

I have seen other people do this already though so I know it's possible even if the results aren't always perfect. But where can I find these types of tools?

$\endgroup$
1
$\begingroup$

In the timespan from 2017-2018 many new papers about the topic "text-to-image generation" were published at Google Scholar. Most of them describing examples from dedicated neural network challenges. Software which is able to do the wished task is called Stackgan, Attngan and Chatpainter. The reason why this success was possible has to do with standardized dataset in which images plus textual annotation are given. What the neural network has to do is to find a connection between them. That means, 100% of the submissions are working with deeplearning and machine learning.

From the technology itself, it is similar to a OCR challenge but in the opposite direction. Instead of recognizing that the image shows a number or a symbol, the task to generate such a image. Usually this is realized with Generative Adversarial Networks.

$\endgroup$
0
$\begingroup$

Well, there is this tool https://brandmark.io/ that can make logos in a bunch of different styles however you wish. I heard about this great tool from Siraj Raval if you are interested in the startup he made last week with brandmark.

$\endgroup$
-2
$\begingroup$

You may wish to rephrase the question because of the don't-ask-for-software tradition brought into the AI SE from Stack Overflow, but as somewhat of a liberty and justice person, I will answer anyway.

There are many tools, APIs, approaches, and technologies to generate images from text, but it is grossly immature. If we give an artist a commission, the specification is usually in textual or vocal form, and there may be dialog so that both parties are clear that the objectives and constraints have been communicated. The field of AI has not reached the point where natural language can ensue between commissioner and AI system such that generation of an image can be as definitive as with human dialog with an experienced fine artist, photographer, graphics artist, or marketing department or firm.

What we are beginning to see is a limited and unreliable comprehension based on a large enough training set of vocabulary and images that correspond to that vocabulary used in sentences that describe the images. Most are based on an equilibrium established between a discriminator and a generator for a set of semantic relationships. This is interesting research because the GAN (generative adversarial network) device is being applied in semantic contexts.

Much of this is proprietary and some is open source but must be cobbled together to make it actually work. To begin down this road, you will not be able (yet) to just download and play. It is a good time to begin understanding this application of multiple AI techniques however. Take a look at GAN, linguistics, and semantic maps and graphs in a scholarly search. Don't combine the three. Learn each separately and then you will see how they must be combined to create what you envision and know what you are doing when you begin downloading libraries and frameworks and start doing your own cobbling together of working systems.

$\endgroup$
  • $\begingroup$ The description with the discriminator and a generator goes into the right direction, but stays only on the surface. Also, the term “set” doesn't fit in this context. $\endgroup$ – Manuel Rodriguez Mar 5 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.