I'm very new to AI. I read somewhere that AI can be used to create GUI UI/UX design. That has fascinated me for a long time. But, since I'm very new here, I don't have any idea how it can happen.

The usual steps to create the UI Design are:

  • Create Grids.
  • Draw Buttons/Text/Boxes/Borders/styles.
  • Choose Color Schemes.
  • Follow CRAP Principle (Contrast, Repeatition, Alignment, Proximity)

I wonder how can AI algorithms help with that. I know a bit of neural networking and the closest I can think of is the following two methods (Supervised Learning).

  1. Draw grids manually and train the Software manually to learn proper styles until it becomes capable of giving modern results and design its own design language.

  2. Take a list of a few websites (for example) from the internet and let the software learn and explore the source code and CSS style sheets and learn and program neurons manually until it becomes capable of making it's own unique styles.


The layout problem is suitable for a rule-based approach, as used to generate levels in Rogue or customised home-layouts, you encode some constraints and search the remaining space. The colour problem is also suitable for a rule-based approach, colour is a 3-dimensional space (4 if you include opacity). A palette can be created through taking colours at appropriate distances from a starting point.

Seen as this domain is well analysed, there are known heuristic and encodable rules. You'll likely have more success and faster by using these to generate designs.

Just to give an idea, seen as rule based approach isn't discussed much these days, the following is valid Prolog, query with phrase(webpage, Design)., that'll quickly generate designs like: [brand, header_nav, searchbox, sidebar_nav, form_content, sidebar_list]. This is just a super-quick demo, you'd likely want a tree instead of a list and you'll need to derive HTML and CSS from it too, which is simple enough in Prolog.

webpage --> body.
webpage --> header, body.
webpage --> header, body, footer.

header --> [brand], header_content.
header_content --> [header_nav].
header_content --> [header_nav], [searchbox].

body --> content.
body --> sidebar(_), content.
body --> { dif(A, B) }, sidebar(A), content, sidebar(B).
body --> content, sidebar(_).

content --> [form_content].
content --> [article_content].
content --> [list_content].

sidebar(list) --> [sidebar_list].
sidebar(nav) --> [sidebar_nav].
sidebar(ads) --> [sidebar_ads].

footer --> [notification].

If you see the use case, on higher level it seems to generate some visual output - the design but when seen at lower level, this design is output of some code.

One way we can do it is to train a neural network that learns to generate code which can be seen as some form of organized text. So now can be treated as a text generation problem on which you can find a lot of literature.

pix2code is one implementation that uses this idea and even expands it. In this paper the authors took it to another level, they also used the visual part - the GUI and built an architecture that takes both the code and the GUI as input and learns to generate code. So eventually it would be capable of generating code when a simple GUI was given.

Also there are some implementations where even the network could produce code even when a rough sketch was given.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy