Skip to main content

Research Repository

Advanced Search

Reproducibility of experiments in recommender systems evaluation

Polatidis, Nikolaos; Kapetanakis, Stelios; Pimenidis, Elias; Kosmidis, Konstantinos

Authors

Nikolaos Polatidis

Stelios Kapetanakis

Konstantinos Kosmidis



Contributors

Lazaros Iliadis
Editor

Ilias Maglogiannis
Editor

Vassilis Plagianakos
Editor

Abstract

© IFIP International Federation for Information Processing 2018 Published by Springer International Publishing AG 2018. All Rights Reserved. Recommender systems evaluation is usually based on predictive accuracy metrics with better scores meaning recommendations of higher quality. However, the comparison of results is becoming increasingly difficult, since there are different recommendation frameworks and different settings in the design and implementation of the experiments. Furthermore, there might be minor differences on algorithm implementation among the different frameworks. In this paper, we compare well known recommendation algorithms, using the same dataset, metrics and overall settings, the results of which point to result differences across frameworks with the exact same settings. Hence, we propose the use of standards that should be followed as guidelines to ensure the replication of experiments and the reproducibility of the results.

Citation

Polatidis, N., Kapetanakis, S., Pimenidis, E., & Kosmidis, K. (2018). Reproducibility of experiments in recommender systems evaluation. IFIP Advances in Information and Communication Technology, 519, 401-409. https://doi.org/10.1007/978-3-319-92007-8_34

Journal Article Type Conference Paper
Conference Name 14th International Conference on Artificial Intelligence Applications and Innovations
Conference Location Rhodes, Greece
Acceptance Date Apr 11, 2018
Online Publication Date May 22, 2018
Publication Date Jan 1, 2018
Deposit Date May 16, 2018
Publicly Available Date Mar 29, 2024
Journal IFIP Advances in Information and Communication Technology
Print ISSN 1868-4238
Publisher Springer Verlag (Germany)
Peer Reviewed Peer Reviewed
Volume 519
Pages 401-409
Series Title IFIP Advances in Information and Communication Technology
DOI https://doi.org/10.1007/978-3-319-92007-8_34
Keywords recommender systems, evaluation, reproducibility, replication
Public URL https://uwe-repository.worktribe.com/output/869675
Publisher URL https://doi.org/10.1007/978-3-319-92007-8_34

Files





You might also like



Downloadable Citations