Dr Paul Matthews Paul2.Matthews@uwe.ac.uk
Senior Lecturer in Information and Data Science
Human-in-the-loop topic modelling: Assessing topic labelling and genre-topic relations with a movie plot summary corpus
Matthews, Paul
Authors
Abstract
A much-used but not yet mainstream text analysis approach, topic modelling allows the identification of lexical themes for a document collection. Against principles for interpretable AI and sociotechnical design, there are definite strengths from its speed and ability to discover structure, but remain challenges in how results can be interpreted whether this be by analysts, domain experts, or potential end users. Automated coherence and labelling measures go some of the way toward bridging the understanding and trust gap, and user empowerment through visualisation and design intervention is starting to show how the remaining ground might be made up. This study uses topic modelling on a corpus of Wikipedia movie summaries to illustrate challenges and potential. Topic labelling for naive users was found to only be easy in a quarter of cases, and difficulty increased markedly with 100 topics compared to 50. While automated measures suggested 88 topics, the number manageable by users was closer to 50. The unsupervised topic model was compared to the movie genre labels and indicated that the two might work together well to complement genres, match content across genre and highlight within-genre variability. It is suggested that unsupervised models might work better for creativity and discovery than semi-supervised versions.
Citation
Matthews, P. Human-in-the-loop topic modelling: Assessing topic labelling and genre-topic relations with a movie plot summary corpus. In ISKO-UK Conference: The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organisation
Conference Name | ISKO-UK Conference: The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organisation |
---|---|
Conference Location | City University, London |
Start Date | Jul 15, 2019 |
End Date | Jul 16, 2019 |
Deposit Date | Feb 14, 2020 |
Publicly Available Date | Feb 14, 2020 |
Book Title | ISKO-UK Conference: The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organisation |
ISBN | 9783956505492 |
Keywords | text analytics, AI, machine learning, text mining |
Public URL | https://uwe-repository.worktribe.com/output/5208861 |
Publisher URL | https://www.iskouk.org/event-3131559 |
Files
ISKO-UK 2019 P Matthews
(2.7 Mb)
PDF
Copyright Statement
This is the accepted version of the conference paper.
You might also like
Development and evaluation of a mobile app to support reading education
(2023)
Conference Proceeding
Engagement with optional formative feedback in a portfolio-based digital design module
(2023)
Journal Article
Genre analysis of movies using a topic model of plot summaries
(2021)
Journal Article
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search