1
$\begingroup$

I am working on implementing an RL agent and I want to demonstrate its effectiveness over a bounded problem space. The setting is essentially a queueing network and so it can be represented as a graph. I want to consider the agent's performance over all graphs up to order $n$ and with average degree from $0$ (edgeless) to $n-1$ (fully connected).

I have looked into generating random graphs using the Erdős–Rényi model, for example. My thought is that I could show the average performance of my agent for different settings of number of nodes and edge probability (under this particular graph generation model).

Are there any established techniques that are along the lines of this approach?

Improve this question
$\endgroup$
  • $\begingroup$ It's not clear to me what you want to represent as a graph (and how exactly do you want to do it? i.e. what are the nodes and edges of this graph?), probably because I don't know what you mean by "queueing network". Moreover, what do you mean by "bounded problem space"? What is bounded here? Finally, I don't understand why you want to evaluate the agent's performance over all graphs? What graphs are you talking about? $\endgroup$ – nbro♦ Oct 20 '20 at 21:09
  • $\begingroup$ Thanks for the link. But I think it would be helpful for those not familiar with queuing networks if you described more explicitly your state space, the action space (remember that, in RL, you typically assume that your problem/environment can be modelled as a Markov decision process? That's why I am asking this info), what RL algorithm you are using, and why your problem is bounded, and why this information is important. $\endgroup$ – nbro♦ Oct 20 '20 at 21:21
  • $\begingroup$ I think the root of my question is independent of my particular MDP formulation. Say we forget the queueing network example and consider a grid world problem (which is probably more familiar to most). I can define a specific grid world instance, with some positive reward positions and some negative, and show that my agent does well most of the time when faced with this specific instance. What I am interested in is showing that my agent does well on many different grid world instances up to a certain grid size, and with various densities of positive and negative reward states. $\endgroup$ – hoffee Oct 20 '20 at 21:34
  • 2
    $\begingroup$ I know someone that is working on a very similar problem. Unfortunately, I can't share with you the details. However, to answer your question "Are there many approaches trying to do a similar thing?", I would say "no", so you may want to investigate your own approaches or try something new. $\endgroup$ – nbro♦ Oct 20 '20 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.