This is a very basic question. I'm running a faster rcnn trainer on a dataset for object recognition. My images range from 200x200 to 7360x4912 in resolution. There are only 2 classes being trained (both are very similar, but slightly different). 2291 total images, 727 labels for one class and 917 labels for the second. After about 400k steps this is what my loss curve looks like. It hasn't completely plateaued just yet, but I'm not convinced it will get to where I need it. My question is about the jaggedness of the results. I have smoothing set to 0.99 to get an overall view of the progress, but the loss actually fluctuates very wildly. Is this to be expected or normal? If not then what should I be looking for to improve the results? Is this mostly a dataset cleanup issue or could the network settings use some fine tuning? Thanks for any help.
There is actually no definition for a perfect loss curve. It varies according to the dataset, the classification problem, the dropout rate, the learning rate and the optimizer used. The loss should not decrease suddenly , which might be a sign of overfitting. It also should not take long time to decrease, or else it will take years to converge.