3
$\begingroup$

Let's say we have a captcha system that consists of a greyscale picture (of a part of a street or something akin to re-captcha), divided into 9 blocks, with 2 missing pieces.

You need to choose the appropriate missing pieces from over 15 possibilities to complete the picture.

The puzzle pieces have their edges processed with glitch treatment as well as they have additional morphs such as heavy jpeg compression, random affine transform, and blurred edges.

Every challenge picture is unique - pulled from a dataset of over 3 million images.

Is it possible for the neural network to reliably (above 50%) predict the missing pieces? Sometimes these are taken out of context and require human logic to estimate the correct piece.

The chance of selecting two answers in correct order is 1/15*1/14.

Improve this question
$\endgroup$
2
  • $\begingroup$ You're thinking of making the task visually more challenging, which ultimately isn't going to help much. I think captcha uses other variables like mouse tracking to discern human movement from an AI-based decision. $\endgroup$
    – Frans Rodenburg
    Jul 7, 2020 at 7:28
  • $\begingroup$ Related: stackoverflow.com/a/27299487/8770170 $\endgroup$
    – Frans Rodenburg
    Jul 9, 2020 at 23:53

2 Answers 2

Reset to default
0
$\begingroup$

Well to give you a short answer, I would say that YES, it would be MORE resistant than a more standard captcha approach...

This being said, I would still go as far as to predict something like a 75-80% successful prediction rates, for a custom model which is designed specifically for defeating a mechanism such as what you describe. The reason why I am fairly confident in such an appraisal, is primarily because of the following:

  1. New techniques which researchers have begun to explore, which are intended to be "Structure Preserving Convolutions" which utilize a higher dimensional filter to store the extra correlation data.

  2. I think that the obfuscation efforts that you mention will definitely help to some degree, although they can be easily defeated by training the model on a dataset which you pull out some portion of the samples during pre-processing and inject the same sort of noise and glitch treatments, etc.

    • An idea that would be worth exploring would be to process your dataset with an adversarial model which you could then use to generate Adversarial Noise that could then be fed into a pre-process step for your images and replace (or extend) the obfuscation efforts!

TL;DR: If you cant beat 'em, then join 'em! Just train a model to defeat your captcha implementation, and then use the model to generate adversarial examples and then apply obfuscations to your data set accordingly!

For more information on what I am talking about in my suggestion for further obfuscation efforts explore some of the papers you can find on Google Scholar - Ensemble Adversarial Training Examples

Improve this answer
$\endgroup$
9
  • 1
    $\begingroup$ The problem with use of adversarial approach here is that you have no measure of whether an example is still easily solved by humans. Without a suitable loss function, you would need to sample test large numbers of outputs with humans, looking for high percent human solvers (and it will need to be very high to be a useful CAPTCHA), and low percent machine solvers. $\endgroup$
    – Neil Slater
    Jul 4, 2020 at 18:56
  • $\begingroup$ yeah but what about universal adversarial perturbations? $\endgroup$
    – user38359
    Jul 5, 2020 at 14:22
  • $\begingroup$ I find that to be a very weak argument against my answer. If you would investigate the literature on adversarial training, you would find that with most adversarial inputs the image is not even visually altered from a human point of view, although completely defeats the ML models attempting to process them. We are talking about slight changes in pixel color values, tiny perturbations, etc. $\endgroup$
    – Chaplin Marchais
    Jul 6, 2020 at 4:20
  • 1
    $\begingroup$ very specific claims and numbers are cited without any justification. it is up to you to justify your claims, of which no part of your answer provides asides from a few name drops and no explanation as to why these techniques might suffice towards this goal. $\endgroup$
    – k.c. sayz 'k.c sayz'
    Jul 6, 2020 at 22:45
  • 1
    $\begingroup$ also your idea of training an adversarial network, though laudable in theory, is ignorant towards the simple fact that adversarial networks training doesn't very often converge towards a stable solution. $\endgroup$
    – k.c. sayz 'k.c sayz'
    Jul 6, 2020 at 22:57
0
$\begingroup$

This is not resistant at all. A simple comparison on the similarity of edge pixels between borders should be very sufficient to break this method completely.

We can do a very simple calculation. Assume the picture is 8bit black and white with each border being 50x50 pixels. Also assume the distribution is continuously uniform between 0-255 (should probably be normally distributed, but whatever). You have a total of 200 pixels that border each other in between borders. Assume that the naturally generated image is continuous in brightness with respect to dimension in at least 10% of the image with +/- 10 units of brightness is acceptable. Thus we have 20 pixels to work with.

In the case where the image is incorrect, we assume the pixel brightness to be iid in [0-255], thus giving us roughly a 8% (21/256) chance of each pixel around the border to be of acceptable similarity. Which gives us about a 10^-22 chance of this algorithm being fooled. You might disagree with my assumed parameters, but to be frank I am probably being too generous in estimating a lower bound.

There are two lessons here: 1. Just because you and others can't think of a way to break your secure system doesn't mean it's actually secure. 2. Modern ML techniques are not strictly stronger than handcrafted algorithms, though I would also imagine that a simple NN would be able to solve this problem easily.

Improve this answer
$\endgroup$
9
  • $\begingroup$ 1. The edges are processef with glitch treatment. That alone is enough to make tjis attack unfeasible. we ar every well aware of this method. $\endgroup$
    – user38359
    Jul 5, 2020 at 18:56
  • $\begingroup$ 2. The decoys are chosen on the basis of similarity of the lightness and the similarity of the pixels around the edges. $\endgroup$
    – user38359
    Jul 5, 2020 at 18:57
  • $\begingroup$ 1. too much noise and no human subject will be able to recognize the object. even if we increase the threshold of noise to something ridiculous like 20% we are still talking about a false positive rate of 10^-14. 2. the identical calculation will demonstrate that the chance of finding a natural scene with such parameters will be something similar. did you even do any math before you make a claim such as "That alone is enough to make tjis attack unfeasible"? there is a good reason why this method is not used even historically (because it doesn't work) $\endgroup$
    – k.c. sayz 'k.c sayz'
    Jul 6, 2020 at 22:53
  • $\begingroup$ if you don't believe me, go ahead and run your systems with this method of security $\endgroup$
    – k.c. sayz 'k.c sayz'
    Jul 6, 2020 at 22:54
  • $\begingroup$ ieeexplore.ieee.org/abstract/document/5692499 $\endgroup$
    – user38359
    Jul 8, 2020 at 7:04

You must log in to answer this question.