# How to recognize sequence of digits in an image

I am learning to program neural networks and others, and I would like to know how I can get the numbers that are in an image, for example if I pass an image that has 123 written, get with my model that there are 123 written, I have tried to use PyTesseract is not very precise, and I would like to do it with a neural network, my current code is quite simple, it recognizes the digits of the mnist dataset such that:

import tensorflow as tf
from tensorflow.keras import Sequential, optimizers
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
import matplotlib.pyplot as plt

mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

print('train_images.shape:', train_images.shape)
print('test_images.shape:', test_images.shape)
plt.imshow(train_images[0])

train_images = train_images.reshape((60000, 28, 28, 1))
test_images = test_images.reshape((10000, 28, 28, 1))

train_images = train_images.astype('float32') / 255
test_images = test_images.astype('float32') / 255

train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)

model = Sequential()

model.add(Conv2D(32, (5, 5), activation = 'relu', input_shape = (28, 28, 1)))

model.add(Conv2D(64, (5, 5), activation = 'relu'))

model.summary()

model.compile(loss = 'categorical_crossentropy', optimizer = 'sgd', metrics = ['accuracy'])

model.fit(train_images, train_labels, batch_size = 100, epochs = 5, verbose = 1)

test_loss, test_accuracy = model.evaluate(test_images, test_labels)

print('Test accuracy:', test_accuracy)


but I would need to know how I can pass an image with a sequence of digits to it, and that it recognizes the digits in question, does anyone know how I could do it? Thanks a lot.

• I don't know exactly what you're looking for and I didn't read your post carefully, but have you heard of optical character recognition? If not, that's probably something you want to look into. – nbro Jan 25 at 12:29
• @nbro Yes, it's what I'm using right now, but I can't make it very accurate – John Doe Jan 26 at 17:23
• @JohnDoe your code is image classification, won't do object detection of digits – datdinhquoc Jan 27 at 8:14

Your task is text recognition, however your code is for classification task. So you need to use different approach for that. You mentioned that you're going to give model 123 and get 123. But you can not do that with just convolutional networks. Images with text are sequential, so you need to use CRNN(Convolutional-Recurrent-Neural-Networks), LSTM(Long-Short-Term-Memory), BiLSTM(Bidirectional-LSTM). In most of the research papers there are convolutional networks are being used just for feature extraction stage. For prediction stage they used recurrent units, such as LSTM cells or RNN.