3
$\begingroup$

Does anyone know, or can we deduce or infer with high probability from its characteristics, whether the neural network used on this site

https://quickdraw.withgoogle.com/

is a type of convolutional neural network (CNN)?

$\endgroup$
6
$\begingroup$

I believe they don't use CNNs. The most important reason why it's because they have more information than a regular image: time. The input they receive is a sequence of (x,y,t) as you draw on the screen, which they refer as "ink". This gives them the construction of the image for free, which a CNN would have to deduce by itself.

They tried two approaches. Their currently most successful approach does the following:

  • Detect parts of the ink that are candidates of being a character
  • Use a FeedForward Neural Network to do character recognition on those candidates
  • Use beam search and a language model to find most the most likely combination of results that results into a word

Their second approach is using an LSTM (a type of Recurrent Neural Network) end-to-end. In their paper they say this was better in a couple languages.

Source: I was an intern in Google's handwriting team in summer 2015 (on which I believe quickdraw is based), but the techniques I explained can be found in this paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.