My DQN model outputs the best traffic light state in an intersection. I used different values of batch size and learning rate to find the best model. How would I know if I got the optimal hyperparameter values?
1 Answer
$\begingroup$
$\endgroup$
6
If possible, I would try to calculate what the (theoretical) maximum throughput through the intersection is for a given time interval. If the control behavior that the DQN produces comes empirically close to the maximally possible throughput score, the model is good. Otherwise, you could measure and compare the throughput of different models and choose the best performing one.
-
$\begingroup$ What does "throughput through the intersection is for a given time interval" actually mean in this context? What is a "throughput score"? $\endgroup$– nbroJun 21, 2020 at 22:24
-
$\begingroup$ Each intersection has some limitations with respect to how many cars can cross it eg per hour. After a certain point, there can just not go 'more' cars through it per time interval, eg due to speed limitations. Given reasonable simulation assumptions, the maximal number of cars that can go through the intersection per hour can be calculated. This measure can then serve as your gold standard to which you compare the results obtained from your DQN controller. Unless, you want to evaluate it in a static scene, which seems odd to me to do, since it wouldn't assess the controller's dynamics. $\endgroup$ Jun 21, 2020 at 22:48
-
$\begingroup$ Some simulation packages that could come handy (also for training DQN on Traffic light control, btw): sumo.dlr.de/pdf/dkrajzew_MESM2002_SUMO.pdf matsim.org carla.org $\endgroup$ Jun 21, 2020 at 23:02
-
$\begingroup$ Well, to be honest, the details of the OP's problem and model aren't clear because the part "My DQN model outputs the best traffic light state in an intersection" isn't very explicit and descriptive enough. Anyway, if I understand you correctly, your suggestion is to get some kind of constraint for each situation. $\endgroup$– nbroJun 21, 2020 at 23:56
-
$\begingroup$ DQN is a Reinforcement Learning algorithm which is trained on solving a particular task in a given environment. Regardless of how ambiguous the original question is, for testing whether DQN does a good job or not, it is not enough to give it e.g. 1000 scene images and consider what the output is, eg because you could assess the respective output traffic light state, but not how long it would decide to remain in it. Therefore, you won't see whether traffic congestion is gonna kick in after some time of letting the controller run (sign for a bad controller) or not (sign for good controller). $\endgroup$ Jun 22, 2020 at 7:29