Evaluation of Multi-Agent Deep Reinforcement Learning Model with Fault-tolerance Attention Mechanism for Traffic Light Control System
Abstract
Managing urban traffic at intersections is a complex challenge. Traditional traffic signal systems struggle to adapt to real-time congestion and variable vehicle flow, particularly at roads with high traffic volume. These systems also lack coordination between neighbouring intersections, leading to inefficient vehicle movement, delays for emergency vehicles, and unsafe pedestrian crossings. This paper proposes a solution using Multi-Agent Reinforcement Learning (MARL) to model a traffic network as a multi-agent system. Specifically, it employs Fault-Tolerant Attention Multi-Agent Deep Deterministic Policy Gradient (FT Attn. MADDPG), where decisions are based on average queue lengths. The Fault-tolerance Attention mechanism allows agents to minimize the impact of malfunctioning agents, improving overall performance. The approach also supports various intersection types through a parametric action space. Simulation results show that FT Attn. MADDPG significantly reduces travel time by 16.21% under high, 26.97% under medium, and 6.89% under low traffic demand compared to standard MADDPG.
Downloads
References
Abdoos, M., Mozayani, N., & Bazzan, A. L. C. (2011). Traffic Light Control in Non-stationary Environments based on Multi Agent Q-learning. 1580–1585.
Azad-Manjiri, M., Afsharchi, M., & Abdoos, M. (2025). DDPGAT: Integrating MADDPG and GAT for optimized urban traffic light control. IET Intelligent Transport Systems, 19(1), 1–20. https://doi.org/10.1049/itr2.70000
Chu, T., Wang, J., Codecà, L., & Li, Z. (2020). Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems, 21(3), 1086–1095. https://doi.org/10.1109/TITS.2019.2901791
Fan, L., Yang, Y., Ji, H., & Xiong, S. (2025). Optimization of Traffic Signal Cooperative Control with Sparse Deep Reinforcement Learning Based on Knowledge Sharing. Electronics 2025, 14(1), 156, 1–19.
Gu, S., Geng, M., & Lan, L. (2021a). Attention-based fault-tolerant approach for multi-agent reinforcement learning systems. Entropy, 23(9). https://doi.org/10.3390/e23091133
Gu, S., Geng, M., & Lan, L. (2021b). Attention-based fault-tolerant approach for multi-agent reinforcement learning systems. Entropy, 23(9), 1– 15. https://doi.org/10.3390/e23091133
Guastella, D. A., & Bontempi, G. (2023). Traffic Modeling with SUMO: a Tutorial. 1–23. http://arxiv.org/abs/2304.05982
Hu, T. Y., & Li, Z. Y. (2024). A multi-agent deep reinforcement learning approach for traffic signal coordination. IET Intelligent Transport Systems, 18(8), 1428– 1444. https://doi.org/10.1049/itr2.12521
Islam, F., Ball, J. E., & Goodin, C. T. (2024). Enhancing Longitudinal Velocity Control With Attention Mechanism-Based Deep Deterministic Policy Gradient (DDPG) for Safety and Comfort. IEEE Access, 12(March), 30765– 30780. https://doi.org/10.1109/ACCESS.2024.3368435
Jin, Q. (2024). Automatic Control of Traffic Lights at Multiple Intersections Based on Artificial Intelligence and ABST Light. IEEE Access, 12(June), 103004– 103017. https://doi.org/10.1109/ACCESS.2024.3433016
Kolat, M., Kővári, B., Bécsi, T., & Aradi, S. (2023). Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach. Sustainability (Switzerland), 15(4). https://doi.org/10.3390/su15043479
Li, Z., Xua, C., & Guohui Zhang. (2021). A Deep Reinforcement Learning Approach for Traffic Signal Optimization. 2021-Septe, 2512–2518. https://doi.org/10.1109/ITSC48978.2021.9564847
Li, Z., Yu, H., Zhang, G., Dong, S., & Xu, C. (2021). Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning. Transportation Research Part C, 125(March), 103059. https://doi.org/10.1016/j.trc.2021.103059
Shi, Y., Pei, H., Feng, L., Zhang, Y., & Yao, D. (2024). Towards Fault Tolerance in Multi-Agent Reinforcement Learning. ArXiv:2412.00534v1 [Cs.LG] 30 Nov 2024, 1(c), 1–14. http://arxiv.org/abs/2412.00534
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489. https://doi.org/10.1038/nature16961
Van Der Pol, E., & Oliehoek, F. A. (2016). Coordinated Deep Reinforcement Learners for Traffic Light Control. 30th Conference on Neural Information Processing Systems (NIPS), Nips, 1–8.
Wei, X., Cui, W. P., Huang, X., Yang, L. F., Tao, Z., & Wang, B. (2023). Graph MADDPG with RNN for multiagent cooperative environment. Frontiers in Neurorobotics, 17(2010). https://doi.org/10.3389/fnbot.2023.1185169
Wiering, M. (2000). Multi-Agent Reinforcement Learning for Traffic Light Control. Proc Intl Conf Machine Learning, JANUARY 2000, 1151– 1158. http://igitur- archive.library.uu.nl/math/2007-0330-200425/wiering_00_multi.pdf
Copyright (c) 2025 James Adunya Omina, Peter Wagacha Waiganjo, PhD, Lawrence Muchemi, PhD, Nicodemus Aketch Ishmael, PhD
This work is licensed under a Creative Commons Attribution 4.0 International License.
