Leveraging LLMs for smart cities qualitative data analysis

Covato, Elisa; Soomro, Kamran; Khan, Zaheer; Bilal, Muhammad; Kirby, Sam; Yiangou, Tom

Abstract

Public authorities regularly conduct surveys and gather data from their citizens. The data is often analysed manually and consumes significant effort. This paper explores how Generative Artificial Intelli- gence (GAI) can help public authorities in automating the data analysis. In this respect we investigate the potential of Large Language Models (LLMs) to perform sentiment analysis and summarisation of the unstruc- tured data as smart services. We used data from the East Bristol Liveable Neighbourhood (EBLN) as a case study. We assess the accuracy and pre- cision of these models and validate the outcomes against ground truth and expert validation. Our results indicate that sentiment classification accuracy is above 90%. In contrast, the expert validation ranked the summarisation without context accuracy highly satisfactory, however, the satisfaction was low when contextual summarisation is evaluated. These findings demonstrate that LLMs are a promising new technique to enhance qualitative analysis efficiency, but more research needs to be conducted to improve accuracy and utility.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: IDC 2024 (17th International Symposium on Intelligent Distributed Computing 2024)
• Start Date: Sep 18, 2024
• End Date: Sep 20, 2024
• Acceptance Date: Aug 26, 2024
• Deposit Date: Jul 18, 2024
• Peer Reviewed: Peer Reviewed
• Public URL: https://uwe-repository.worktribe.com/output/12690289
• Related Public URLs: https://blogs.brighton.ac.uk/idc2024/