I want to compare the time complexity of two deep neural networks, but I have no idea how to go about it. How do I graphically achieve that with respect to the number of iterations, accuracy or any other metric?
Time complexity in Big-O doesn't make much sense as it is dominated by matrix multiplication.
There are three things you can do:
- Calculate the operations necessary for one training step
- Measure time one training step (average!!!)
- Measure time for training until a certain accuracy / error / score is reached
The last one is practically the most relevant.
Don't forget to mention hardware (could+GPU) and software (drivers, cuda, cudnn; all with versions) being used - that makes a huge difference
See tables in the appendix of https://arxiv.org/abs/1707.09725 to get some "design performance numbers" of famous architecture