Assessing a decision support tool for SOC analysts

Happa, Jassim; Helmhout, Martin; Philips, Royal; Thomas Bashford-Rogers, N L; Goldsmith, Michael; Creese, Sadie; Happa, Jassim; Agrafiotis, Ioannis; Helmhout, Martin; Bashford-Rogers, Thomas; Goldsmith, Michael

Abstract

It is difficult to discern real-world consequences of attacks on an enterprise when investigating network-centric data alone. In recent years, many tools have been developed to help understand attacks using visualization, but few aim to predict real-world consequences. We have developed a visualization tool that aims to improve decision support during attacks in Security Operation Centres (SOCs). Our tool visualizes propagation of risks from sensor alert data to Business Process (BP) tasks. This is an important capability gap present in many SOCs today as most threat detection tools are technology-centric. In this paper we present a user study that assesses our tool's usability and ability to support the analyst. Ten analysts from seven SOCs performed carefully designed tasks related to understanding risks and recovery decision-making. The study was conducted in laboratory conditions with simulated attacks and used a mixed-method approach to collect data from questionnaires, eye tracking and semi-structured interviews. Our findings suggest that relating business tasks to network asset in visualizations can help analysts prioritise response strategies. Finally, our paper also provides an in-depth discussion on user studies conducted with SOC analysts more generally, including lessons learnt, recommendations and a critique of our own study.

• Journal Article Type: Article
• Acceptance Date: Oct 25, 2020
• Deposit Date: Jan 8, 2021
• Journal: Digital Threats: Research and Practice
• Print ISSN: 2692-1626
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Volume: 9
• Issue: 4
• Article Number: 39
• Keywords: CCS Concepts: • Human-centered computing → Empirical studies in visualization; Empirical studies in interaction design; Visual analytics; • Security and privacy → Intrusion detection systems; Network security; Additional Key Words and Phrases: Bus
• Public URL: https://uwe-repository.worktribe.com/output/6974935