I am trying to train an OCR model with Vision Transformers. While training the output is a vector with values full of zeros which is obviously padding value. But the CTC loss was small that it was negligible. For example, the expected output predicted output, and loss I got were:-
Predicted: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], device='cuda:0') Expected: tensor([ 2, 913, 310, 888, 310, 852, 310, 3, 0, 0], device='cuda:0') Loss 9.179017069982365e-06
The length of the predicted vector is 128, but I have only printed the first 10 values here. None of the values match here. but the loss as you can see is incredibly low. Interesting thing is that I have used the exact same loss function with CNN based OCR and I have gotten good results.