I am working on implementing an RL agent and I want to demonstrate its effectiveness over a bounded problem space. The setting is essentially a queueing network and so it can be represented as a graph. I want to consider the agent's performance over all graphs up to order $n$ and with average degree from $0$ (edgeless) to $n-1$ (fully connected).
I have looked into generating random graphs using the Erdős–Rényi model, for example. My thought is that I could show the average performance of my agent for different settings of number of nodes and edge probability (under this particular graph generation model).
Are there any established techniques that are along the lines of this approach?