For object detection tasks I have a few minutes of video footage from a surveillance camera, converted to a sequence of images and ground truth bounding boxes for all people walking by.
Now what's the best way to split this into training, validation and test sets (80/10/10)?
- I could randomly select 10% for testing and 10% for validation and rest into training.
- The first 80% go into training, next 10% to validation, rest to testing.
The first way has the advantage of having a good distribution of different people walking by and also more varying densities and locations of people in the test set. But the disadvantage would be that for each testing image a very similar image exists in the training set.
The second way would have the advantage of the testset being more truly "never seen" during training, but at cost of less variety.