Multi-agent learning of asset maintenance plans through localised subnetworks

Pérez Hernández, Marco; Puchkova, Alena; Parlikad, Ajith Kumar

Abstract

Maintenance planning of networked multi-asset systems is a complex problem due to the inherent individual and collective asset constraints and dynamics as well as the size of the system and interdependencies among assets. Although multi-asset systems have been studied numerous times in the past decades, maintenance planning implications of the system's network characteristics have been barely analysed. Likewise, solutions that consider the network perspective suffer from scalability issues as a network-wide observability is assumed. This paper proposes a network maintenance planning approach based on the decomposition of the multi-asset network into fixed-size localised subnetworks. The overall network maintenance plan is produced by aggregating the subnetwork maintenance plans, which are computed independently via a multi-agent deep reinforcement learning (MARL) algorithm. The results are evaluated against a network-wide approach as well as the commonly-used individual approach. The paper also introduces a systematic approach to integrate the MARL resulting policy in a multi-asset agent-based model. Simulation results of several random asset networks and a large nationwide network infrastructure show that, although a network-wide approach outperforms, on average, other approaches considered, the localised subnetworks approach, provides an acceptable alternative in networks with small-world properties, without the need of a network-wide view.

• Journal Article Type: Article
• Acceptance Date: Oct 23, 2023
• Online Publication Date: Nov 3, 2023
• Publication Date: Jan 31, 2024
• Deposit Date: Jan 19, 2024
• Publicly Available Date: Jan 24, 2024
• Journal: Engineering Applications of Artificial Intelligence
• Print ISSN: 0952-1976
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 127
• Issue: Part B
• Article Number: 107362
• DOI: https://doi.org/10.1016/j.engappai.2023.107362
• Keywords: agent-based maintenance; network maintenance; multi-agent reinforcement learning
• Public URL: https://uwe-repository.worktribe.com/output/11391140
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S0952197623015464