Examining neural networks through architectural variation analysis for image classification

Liu, Haixia; Brailsford, Tim; Bull, Larry; Goulding, James; Smith, Gavin

Abstract

This paper presents a method for examining neural networks in image classification through architectural variation analysis. Small-scale experiments generate initial insights, and the configurations are further tested on entire datasets. The newly proposed sampling strategy, which focuses on heavily confused samples to identify instance hardness, offers researchers a way to explore novel methods with reduced computational costs. Image pre-processing operations are crucial in the image classification pipeline. Applying image sharpening prior to other standard pre-processing techniques was found to yield improved results. The choice and order of layers significantly impact model performance. We propose three layer-level operations: Plug and Play (PaP), Leave One Layer Out (LOLO), and Select and Reorder (SaRe). The results indicate that convolutional (Conv2D) and batch normalization (BN) layers are significant in image classification tasks, but this is dependent upon the context of the images. Performing BN before Conv2D can improve the model's predictive capability. This study provides valuable insights into optimizing deep learning models, with potential avenues for future research, including explainable AI (XAI).

• Presentation Conference Type: Conference Paper (published)
• Conference Name: International Conference on Information Science and Systems (ICISS 2024)
• Start Date: Aug 14, 2024
• End Date: Aug 16, 2024
• Acceptance Date: Jun 18, 2024
• Online Publication Date: Jan 31, 2025
• Publication Date: Jan 31, 2025
• Deposit Date: Aug 16, 2024
• Publicly Available Date: Mar 1, 2025
• Peer Reviewed: Peer Reviewed
• Pages: 55-61
• Book Title: Proceedings of the 2024 7th International Conference on Information Science and Systems
• ISBN: 9798400717567
• DOI: https://doi.org/10.1145/3700706.3700715
• Keywords: Architectural Variation Analysis; Explainable AI; Layer Selection; Layer Ordering; Deep Learning Optimization; Neural Networks; LOLO
• Public URL: https://uwe-repository.worktribe.com/output/12791966