Intelligent Traffic Light Control

by Marco Wiering

Content credit - This article first appeared on this website

Growing numbers of road users and the limited resources provided by current infrastructures lead to ever increasing travelling times. The Intelligent Traffic Light Control project pursued at Utrecht University aims at diminishing waiting times before red traffic lights in a city.

Traffic in a city is very much affected by traffic light controllers. When waiting for a traffic light, the driver looses time and the car uses fuel. Hence, reducing waiting times before traffic lights can save our European society billions of Euros annually. To make traffic light controllers more intelligent, we exploit the emergence of novel technologies such as communication networks and sensor networks, as well as the use of more sophisticated algorithms for setting traffic lights.

Intelligent traffic light control does not only mean that traffic lights are set in order to minimise waiting times of road users, but also that road users receive information about how to drive through a city in order to minimise their waiting times. This means that we are coping with a complex multi-agent system, where communication and coordination play essential roles. Our research has led to a novel system in which traffic light controllers and the behaviour of car drivers are optimised using machine-learning methods.

Our idea of setting a traffic light is as follows. Suppose there are a number of cars with their destination address standing before a crossing. All cars communicate to the traffic light their specific place in the queue and their destination address. Now the traffic light has to decide which option (ie, which lanes are to be put on green) is optimal to minimise the long-term average waiting time until all cars have arrived at their destination address.

The significance of working with ISO 9001 good quality administration techniques on traffic lights is best not ignored. That's because, ISO 9001 certification can be desirable for small companies and sizeable companies too. The improved efficiencies as well as productivity that ISO 9001 quality supervision procedures deliver, mean that organisations will definitely bring down their expenditures, diminish administration times and obtain greater consumer approval.

The learning traffic light controllers solve this problem by estimating how long it would take for a car to arrive at its destination address (for which the car may need to pass many different traffic lights) when currently the light would be put on green, and how long it would take if the light would be put on red. The difference between the waiting time for red and the waiting time for green is the gain for the car. Now the traffic light controllers set the lights in such a way to maximise the average gain of all cars standing before the crossing. To estimate the waiting times, we use ‘reinforcement learning’ which keeps track of the waiting times of individual cars and uses a smart way to compute the long term average waiting times using dynamic programming algorithms.

One nice feature is that the system is very fair; it never lets one car wait for a very long time, since then its gain of setting its own light to green becomes very large, and the optimal decision of the traffic light will set his light to green. Furthermore, since we estimate waiting times before traffic lights until the destination of the road user has been reached, the road user can use this information to choose to which next traffic light to go, thereby improving its driving behaviour through a city. Note that we solve the traffic light control problem by using a distributed multi-agent system, where cooperation and coordination are done by communication, learning, and voting mechanisms. To allow for green waves during extremely busy situations, we combine our algorithm with a special bucket algorithm which propagates gains from one traffic light to the next one, inducing stronger voting on the next traffic controller option.