Nikolaos Polatidis
A guideline-based approach for assisting with the reproducibility of experiments in recommender systems evaluation
Polatidis, Nikolaos; Pimenidis, Elias; Fish, Andrew; Kapetanakis, Stelios
Authors
Dr Elias Pimenidis Elias.Pimenidis@uwe.ac.uk
Senior Lecturer in Computer Science
Andrew Fish
Stelios Kapetanakis
Abstract
Recommender systems' evaluation is usually based on predictive accuracy and information retrieval metrics, with better scores meaning recommendations are of higher quality. However, new algorithms are constantly developed and the comparison of results of algorithms within an evaluation framework is difficult since different settings are used in the design and implementation of experiments. In this paper, we propose a guidelines-based approach that can be followed to reproduce experiments and results within an evaluation framework. We have evaluated our approach using a real dataset, and well-known recommendation algorithms and metrics; to show that it can be difficult to reproduce results if certain settings are missing, thus resulting in more evaluation cycles required to identify the optimal settings.
Citation
Polatidis, N., Pimenidis, E., Fish, A., & Kapetanakis, S. (2019). A guideline-based approach for assisting with the reproducibility of experiments in recommender systems evaluation. International Journal on Artificial Intelligence Tools, 28(8), Article 1960011. https://doi.org/10.1142/S021821301960011X
Journal Article Type | Article |
---|---|
Acceptance Date | Jul 24, 2019 |
Online Publication Date | Dec 4, 2019 |
Publication Date | Dec 4, 2019 |
Deposit Date | Jan 14, 2020 |
Publicly Available Date | Mar 29, 2024 |
Journal | International Journal on Artificial Intelligence Tools |
Print ISSN | 0218-2130 |
Electronic ISSN | 1793-6349 |
Publisher | World Scientific Publishing |
Peer Reviewed | Peer Reviewed |
Volume | 28 |
Issue | 8 |
Article Number | 1960011 |
DOI | https://doi.org/10.1142/S021821301960011X |
Public URL | https://uwe-repository.worktribe.com/output/3633101 |
Publisher URL | https://www.worldscientific.com/doi/10.1142/S021821301960011X |
Files
A guideline-based approach for assisting with the reproducibility of experiments in recommender systems evaluation
(1.2 Mb)
PDF
Licence
http://www.rioxx.net/licenses/all-rights-reserved
Publisher Licence URL
http://www.rioxx.net/licenses/all-rights-reserved
Copyright Statement
Preprint of an article published in International Journal on Artificial Intelligence Tools, 28, 8, 2019, 1960011 https://doi.org/10.1142/S021821301960011X © [copyright World Scientific Publishing Company] https://www.worldscientific.com/doi/10.1142/S021821301960011X
You might also like
Fast and accurate evaluation of collaborative filtering recommendation algorithms
(2022)
Conference Proceeding
Problem classification for tailored help desk auto replies
(2022)
Conference Proceeding
Supporting patient nutrition in critical care units
(2022)
Conference Proceeding
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search