Skip to main content

Research Repository

Advanced Search

Human-in-the-loop topic modelling: Assessing topic labelling and genre-topic relations with a movie plot summary corpus

Matthews, Paul

Human-in-the-loop topic modelling:  Assessing topic labelling and genre-topic relations with a movie plot summary corpus Thumbnail


Authors



Abstract

A much-used but not yet mainstream text analysis approach, topic modelling allows the identification of lexical themes for a document collection. Against principles for interpretable AI and sociotechnical design, there are definite strengths from its speed and ability to discover structure, but remain challenges in how results can be interpreted whether this be by analysts, domain experts, or potential end users. Automated coherence and labelling measures go some of the way toward bridging the understanding and trust gap, and user empowerment through visualisation and design intervention is starting to show how the remaining ground might be made up. This study uses topic modelling on a corpus of Wikipedia movie summaries to illustrate challenges and potential. Topic labelling for naive users was found to only be easy in a quarter of cases, and difficulty increased markedly with 100 topics compared to 50. While automated measures suggested 88 topics, the number manageable by users was closer to 50. The unsupervised topic model was compared to the movie genre labels and indicated that the two might work together well to complement genres, match content across genre and highlight within-genre variability. It is suggested that unsupervised models might work better for creativity and discovery than semi-supervised versions.

Citation

Matthews, P. Human-in-the-loop topic modelling: Assessing topic labelling and genre-topic relations with a movie plot summary corpus. In ISKO-UK Conference: The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organisation

Conference Name ISKO-UK Conference: The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organisation
Start Date Jul 15, 2019
End Date Jul 16, 2019
Deposit Date Feb 14, 2020
Publicly Available Date Feb 14, 2020
Book Title ISKO-UK Conference: The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organisation
ISBN 9783956505492
Keywords text analytics, AI, machine learning, text mining
Public URL https://uwe-repository.worktribe.com/output/5208861
Publisher URL https://www.iskouk.org/event-3131559

Files

ISKO-UK 2019 P Matthews (2.7 Mb)
PDF

Copyright Statement
This is the accepted version of the conference paper.







You might also like



Downloadable Citations