Solving logical pattern puzzles with Machine Learning?

Question

I found this kind of problem while reading about some web tests companies use to screen applicants. It is a puzzle where you need to guess what comes inside?

Looking at this made me wonder, How would you approach solving problems like this using Machine Learning?

I assume that if people can recognize the pattern with a bit of training, so should A.I. be able to as well.

If anyone knows anything that could be a good starting for me to solve this problem, please let me know. Any opinion will be appreciated! (whether this is possible or not, how difficult it is, computer vision models, algorithms etc., cool projects like this etc.)

I've seen other people solve logic games like here But visual cues get complicated like this often in this type of puzzles.

Answer 1

These specific puzzles are called Raven's Progressive Matrices, a common psychometric tool used to measure the IQ. Knowing that is easy to find references. Here just a few examples, but the literature of course is broader, visual reasoning is quite an old AI topic.

• Similarity-Based Reasoning, Raven’s Matrices, and General Intelligence
• RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
• Solving Raven’s Progressive Matrices with Multi-Layer Relation Networks

From a technical perspective the approaches are not that impressive, classic CNN (and of course more fancy architectures) trained on detecting which answer amongst the given ones is the most "logically" similar to the images forming the incomplete sequence. Notice also that the task is reasonably tractable because we have a set of given answers, without that the story would be much different.

Also, not really related to the question title, but if you're interested in machine leaning and abstract reasoning a much more impressive task to tackle is numerical sequences prediction, and a fresh new paper managed to achieve incredible results in this task as well.