0
$\begingroup$

I have a task to build the Tiktok Manga/Anime AI model or the Anime AI model like Snap Edit application.

I have researched and collected several models on Github, but they still are not good enough, like Tiktok or SnapEdit. These are some models on GitHub:

These models focus on only the face without a background which does not cartoonize the whole image like SnapEdit. This is an example:

enter image description here enter image description here

So could you give me some direction to start with this task?

My idea: I have plan to do following steps, but I belive that it is so tough and take time.

  • Detect each object on the images.
  • Replace each object with a similar anime/manga style image -> Need to compare all images.
  • Put anime objects back into the image and make them fit with the image.
  • Find a similar anime background and replace it too.

It would lead to a problem about how to build the database with anime style? How to find a similar anime/cartoon image? How to get objects and replace them without blur or weird images?

Thank you for helping!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You better use diffusion not GAN. Style transfer isn't enough. You need to re-render the entire landscape to animate like tiktok. Use img2img from this repo.

https://github.com/AUTOMATIC1111/stable-diffusion-webui

If you want a light-weighted model, use this. But you have to manually fine-tune the model into the animal style. I do not recommend using this unless you have a solid ML background

https://huggingface.co/OFA-Sys/small-stable-diffusion-v0

For the dataset, use danbouru dataset. Either from here or small version is available in huggingface or use Grabber

https://gwern.net/danbooru2021#danbooru2020
https://huggingface.co/datasets/tensorcat/Danbooru-2020-Small
https://github.com/Bionus/imgbrd-grabber
Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .