I had an assumption that value iteration will take more iterations to converge if the map size increases/environment's complexity increases.
I tried to verify this idea by running value iteration on randomly generated grid worlds of increasing sizes (from 5X5 grid world, 6x6 grid world ... to 50x50 grid world). In each grid world, there are 4 states with reward value of +1. All other states have reward value of 0, -0.03, -1.
Strangely, I noticed that although the computation time for value iteration to converge increased significantly (which is expected since the time complexity of each iteration increases as the grid world's size increases), number of iterations it took for value iteration to converge remained the same at 66 across all map sizes. This contradicts with my assumption that the number of iterations of value iteration should generally increase if the size/complexity of the environment increases.
I am quite sure my code implementation is correct (I have checked my code and policy/values obtained from the value iterations), and I believe the stop condition that I set for the value iteration is reasonable (I stops value iterations if the maximum change in the values in the current iteration is less than 0.001).
Could anyone explain this phenomenon, and if there are factors that may affect the number of iterations of value iteration?