Skip to main content

Research Repository

Advanced Search

A guideline-based approach for assisting with the reproducibility of experiments in recommender systems evaluation

Polatidis, Nikolaos; Pimenidis, Elias; Fish, Andrew; Kapetanakis, Stelios

Authors

Nikolaos Polatidis

Andrew Fish

Stelios Kapetanakis



Abstract

Recommender systems' evaluation is usually based on predictive accuracy and information retrieval metrics, with better scores meaning recommendations are of higher quality. However, new algorithms are constantly developed and the comparison of results of algorithms within an evaluation framework is difficult since different settings are used in the design and implementation of experiments. In this paper, we propose a guidelines-based approach that can be followed to reproduce experiments and results within an evaluation framework. We have evaluated our approach using a real dataset, and well-known recommendation algorithms and metrics; to show that it can be difficult to reproduce results if certain settings are missing, thus resulting in more evaluation cycles required to identify the optimal settings.

Citation

Polatidis, N., Pimenidis, E., Fish, A., & Kapetanakis, S. (2019). A guideline-based approach for assisting with the reproducibility of experiments in recommender systems evaluation. International Journal on Artificial Intelligence Tools, 28(8), Article 1960011. https://doi.org/10.1142/S021821301960011X

Journal Article Type Article
Acceptance Date Jul 24, 2019
Online Publication Date Dec 4, 2019
Publication Date Dec 4, 2019
Deposit Date Jan 14, 2020
Publicly Available Date Mar 29, 2024
Journal International Journal on Artificial Intelligence Tools
Print ISSN 0218-2130
Electronic ISSN 1793-6349
Publisher World Scientific Publishing
Peer Reviewed Peer Reviewed
Volume 28
Issue 8
Article Number 1960011
DOI https://doi.org/10.1142/S021821301960011X
Public URL https://uwe-repository.worktribe.com/output/3633101
Publisher URL https://www.worldscientific.com/doi/10.1142/S021821301960011X

Files


A guideline-based approach for assisting with the reproducibility of experiments in recommender systems evaluation (1.2 Mb)
PDF

Licence
http://www.rioxx.net/licenses/all-rights-reserved

Publisher Licence URL
http://www.rioxx.net/licenses/all-rights-reserved

Copyright Statement
Preprint of an article published in International Journal on Artificial Intelligence Tools, 28, 8, 2019, 1960011 https://doi.org/10.1142/S021821301960011X © [copyright World Scientific Publishing Company] https://www.worldscientific.com/doi/10.1142/S021821301960011X




You might also like



Downloadable Citations