Nikolaos Polatidis
Reproducibility of experiments in recommender systems evaluation
Polatidis, Nikolaos; Kapetanakis, Stelios; Pimenidis, Elias; Kosmidis, Konstantinos
Authors
Stelios Kapetanakis
Dr Elias Pimenidis Elias.Pimenidis@uwe.ac.uk
Senior Lecturer in Computer Science
Konstantinos Kosmidis
Contributors
Lazaros Iliadis
Editor
Ilias Maglogiannis
Editor
Vassilis Plagianakos
Editor
Abstract
© IFIP International Federation for Information Processing 2018 Published by Springer International Publishing AG 2018. All Rights Reserved. Recommender systems evaluation is usually based on predictive accuracy metrics with better scores meaning recommendations of higher quality. However, the comparison of results is becoming increasingly difficult, since there are different recommendation frameworks and different settings in the design and implementation of the experiments. Furthermore, there might be minor differences on algorithm implementation among the different frameworks. In this paper, we compare well known recommendation algorithms, using the same dataset, metrics and overall settings, the results of which point to result differences across frameworks with the exact same settings. Hence, we propose the use of standards that should be followed as guidelines to ensure the replication of experiments and the reproducibility of the results.
Citation
Polatidis, N., Kapetanakis, S., Pimenidis, E., & Kosmidis, K. (2018). Reproducibility of experiments in recommender systems evaluation. IFIP Advances in Information and Communication Technology, 519, 401-409. https://doi.org/10.1007/978-3-319-92007-8_34
Journal Article Type | Conference Paper |
---|---|
Conference Name | 14th International Conference on Artificial Intelligence Applications and Innovations |
Conference Location | Rhodes, Greece |
Acceptance Date | Apr 11, 2018 |
Online Publication Date | May 22, 2018 |
Publication Date | Jan 1, 2018 |
Deposit Date | May 16, 2018 |
Publicly Available Date | Mar 29, 2024 |
Journal | IFIP Advances in Information and Communication Technology |
Print ISSN | 1868-4238 |
Publisher | Springer Verlag (Germany) |
Peer Reviewed | Peer Reviewed |
Volume | 519 |
Pages | 401-409 |
Series Title | IFIP Advances in Information and Communication Technology |
DOI | https://doi.org/10.1007/978-3-319-92007-8_34 |
Keywords | recommender systems, evaluation, reproducibility, replication |
Public URL | https://uwe-repository.worktribe.com/output/869675 |
Publisher URL | https://doi.org/10.1007/978-3-319-92007-8_34 |
Files
LNCS_Reproducibility_finalVersion.pdf
(299 Kb)
PDF
Licence
http://www.rioxx.net/licenses/all-rights-reserved
Publisher Licence URL
http://www.rioxx.net/licenses/all-rights-reserved
Copyright Statement
This is the accepted version of the paper, published in Artificial Intelligence Applications and Innovations: AIAI 2018
You might also like
Fast and accurate evaluation of collaborative filtering recommendation algorithms
(2022)
Conference Proceeding
Problem classification for tailored help desk auto replies
(2022)
Conference Proceeding
Supporting patient nutrition in critical care units
(2022)
Conference Proceeding
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search