# Markov Model for a Traffic Intersection

I need some help in developing a Markov Model for a crossroads there is no one way road and i am assuming at this time that traffic is only allowed to go straight no turns are allowed. There are 4 roads and 4 signals(agents).

• What are you trying to observe? Markov models are powerful for making observation from a hidden variable. What are you starting states and what is the injected information? – JahKnows Jul 24 '17 at 14:04
• I am trying to find a optimal policy for reducing the waiting time of traffic on signal. and about the sates I am confused i do not know how to start and proceed with multiple agents. – user3751 Jul 24 '17 at 14:11
• If you draw your intersection maybe we can get a better idea of what you mean. Is it a requirement to have multiple agents? If you can restrict that the all of the traffic lights are never set in such a way as to cause accidents (all green), it should reduce to a smaller state space and be easier to consider. – Jaden Travnik Aug 1 '17 at 1:55
• As far i am able to think, you need traffic data for all roads, Based on that you can identify congestion time/traffic using MM/HMM/SSM. Based on traffic at each road at same time, you can optimise the waiting time. – Arpit Sisodia Aug 15 '17 at 5:44
• it is ready now you can try it @ github.com/aawadall/SmartTrafficIntersection it is text based though, you can extend it to have a graphical component to it, or if you want you can port it to whatever language you prefer – A.Rashad Aug 28 '17 at 23:51

Why not use a single agent to control the intersection with the following rules:

• define 8 traffic lights
• Each light has two possible values, 0 or 1, with 0 = Red and 1 = Green
• Lights are as follows:
• 0 = North to South
• 1 = North to East (Left Turn)
• 2 = East to West
• 3 = East to South (Left Turn)
• 4 = South to North
• 5 = South to West (Left Turn)
• 6 = West to East
• 7 = West to North (Left Turn)
• map lights into a bitmap of 8 bits, with the least significant bit (left bit) is the North to south

Given the above, we have the following valid moves, which are both State Space and Action Space :

• State/Move 0 - Vertical Moves only = 10001000
• State/Move 1 - Horizontal Moves only = 00100010
• State/Move 2 - Originated from North only = 11000000
• State/Move 3 - Originated from East only = 00110000
• State/Move 4 - Originated from South only = 00001100
• State/Move 5 - Originated from West only = 00000011
• State/Move 6 - Vertical Diagonal only = 01000100
• State/Move 7 - Horizontal Diagonal only = 00010001

i.e. S = {10001000, 10001000, 00100010, 11000000, 00110000, 00001100, 00000011, 01000100, 00010001}

and A = {10001000, 10001000, 00100010, 11000000, 00110000, 00001100, 00000011, 01000100, 00010001} Now we have aA and sS and at any given s, a will cause the next state s' to have the same value of a

Final component to complete the moel, is the reward signal, which can be simply the negative sum of time spent per car in the intersection

i.e. rt+1 = -∑ciCars time(ci)

a single agent, will be able to optimize the solution to avoid punishment (negative value caused by wait time)

### Update

You can increase the state space to double or triple its size to accommodate for time, by appending a time counter component to the bitmap. in addition to an extra (9th) action of do nothing,

This should give a more subtle traffic controller. e.g. Vertical Traffic Only, at timer = 0 can be represented by 100010000000 or in hex 880, then at time 1 = 881, and so on until time 15 to be 88F.

## Update 2

You can find the source code here and here

• By the way, I started implementing this solution in C#, if you are interested, I can send you an invite to its git repository – A.Rashad Aug 24 '17 at 15:55