I am trying to train an OCR model with Vision Transformers. While training the output is a vector with values full of zeros which is obviously padding value. But the CTC loss was small that it was negligible. For example, the expected output predicted output, and loss I got were:-

Predicted: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], device='cuda:0')
Expected: tensor([  2, 913, 310, 888, 310, 852, 310,   3,   0,   0], device='cuda:0')
Loss 9.179017069982365e-06

The length of the predicted vector is 128, but I have only printed the first 10 values here. None of the values match here. but the loss as you can see is incredibly low. Interesting thing is that I have used the exact same loss function with CNN based OCR and I have gotten good results.



You must log in to answer this question.