How to calculate mean speed in FPS for an object detection model like YOLOv3 or YOLOv3-Tiny? Different object detection models are often presented on charts like this: I am using the DarkNet framework in my project and I want to create similar charts for my own models based on YOLOv3. Is there some easy way to get mean FPS speed for my model with the "test video"?
2 Answers
You can use the dataset test set as "frames" of video. Test the images with your model and calculate the images per second of the result and that is the same as frames per second. However you should set the batch size to 1 as in the real world scenario. You should also display each image with teh corresponding boxes after inference and remove the accuracy calculation as to imitate the real world situation.
@Clement Hui
Thanks for your answer, I ask AlexeyAB from Darknet the same question and he add now flag for Darknet for this type of model speed measurments: https://github.com/AlexeyAB/darknet/issues/4627
I added -benchmark flag for detector demo, now you can use command 2652263
./darknet detector demo obj.data yolo.cfg yolo.weights test.mp4 -benchmark
But for very fast models the bottleneck will be in the Video Capturing from file/camera, >or in Video Showing (you can disable showing by using -dont_show flag).
I think that it is the best solution, you only need the newest version of Darknet (from AlexeyAB).