There are 8 distinct action classes and around 50+ videos per class. I was wondering if flipping videos from the training set can be a good option to generate additional data. Is it?
1 Answer
$\begingroup$
$\endgroup$
3
Probably flipping a video left/right will be OK and useful for your case.
When considering data augmentation approaches, you should think about two things that may prevent it working:
Could the augentation change the label? E.g. would a human looking at the augmented data still label it the same way?
Does the augmentation create data that is too different from expected later use?
So in your case for action recognition you should only be concerned (and maybe not add the left-right flipped videos) if:
The activity label would change depending if tasks were performed left-handed or right-handed.
There is lots of content in the videos (e.g. writing) that is unrealistic when flipped. This is why you should only flip left/right, not top/bottom, in most cases. However, top/bottom and even arbitrary rotation might be fine if your videos are a top-down view, so it depends on your specific case.
As an aside, it is also important to keep the original and augmented copy in the same part of the dataset (training, cross-validation or testing), because they are correlated - not doing so may cause a data leak that will prevent you measuring performance correctly. To play this safe, you should only augment training data, so that you don't risk measuring generalisation of your model against imaginary production data that could not occur in reality.
Other augmentations you might consider for video could be small rotations, random crops, and minor colour, contrast and brightness adjustments.
answered Nov 11, 2021 at 12:34
-
$\begingroup$ Thanks @Neil for the detailed response. My dataset consists road accidents as once class which remains unaffected to the flipping. Also I was using augmentation to flip images horizontally and rotate slightly by 10 degrees. But then, I still was wondering if could chuck the horizontal flip data augmentation and create more data just by flipping them and then add other augmentations for more ruobustness. What do you think? $\endgroup$– Uday SaiNov 11, 2021 at 13:08
-
$\begingroup$ @UdaySai I cannot see your images, even if I could, I don't know what your production scenario is like. I would say your biggest concern is my second point - at what point do your augmentations get too far away from the real dataset that you will be predicting against. One way to play that safe is to augment training data only, not validation or test data. $\endgroup$ Nov 11, 2021 at 15:16
-
$\begingroup$ It is possible that road accidents are impacted by horizontal flipping, since the road system is handed. I would still do the augmentation, because it may not be a big effect compared to other detectable features of accidents. But definitely do not augment cross-validation or test sets. $\endgroup$ Nov 11, 2021 at 15:22