Consistency of XAI models against medical expertise: An assessment protocol

Arnaud, Emilien; Elbattah, Mahmoud; Pitteman, Amandine; Dequen, Gilles; Ghazali, Daniel Aiham; Moreno-Sánchez, Pedro A.

Abstract

Despite the significant advances made by Artificial Intelligence (AI) models in enhancing medical diagnostics and prognostics, their opacity poses a hurdle to widespread clinical adoption. In this regard, Explainable AI (XAI) aims to demystify these complex models, such as neural networks, by revealing the reasoning behind predictions. However, a notable gap exists in enabling non-experts to verify these explanations, necessitating human-in-the-loop evaluation. This paper introduces a systematic protocol, including a novel "consistency" metric, to evaluate the SHAP-based explanations of XAI, comparing them against the clinical knowledge of expert clinicians. We demonstrate how this metric could facilitate both global and feature-specific analyses, operating at the level of individual instances, and thus enhancing AI transparency. It is conceived that the implications of this work may extend beyond the medical context, offering a standardized methodology that could potentially improve the interpretability and acceptance of AI systems in diverse domains.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI)
• Start Date: Jun 3, 2024
• End Date: Jun 6, 2024
• Acceptance Date: Apr 3, 2024
• Online Publication Date: Aug 22, 2024
• Publication Date: Aug 22, 2024
• Deposit Date: Aug 24, 2024
• Publicly Available Date: Aug 27, 2024
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Peer Reviewed: Peer Reviewed
• Pages: 732-736
• Series ISSN: 2575-2634
• Book Title: 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI)
• DOI: https://doi.org/10.1109/ichi61247.2024.00116
• Public URL: https://uwe-repository.worktribe.com/output/12808822