Reproducibility of experiments in recommender systems evaluation

Nikolaos Polatidis, Stylianos Kapetanakis, Elias Pimenidis, Konstantinos Kosmidis

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBN

Abstract

Recommender systems evaluation is usually based on predictiveaccuracy metrics with better scores meaning recommendations of higherquality. However, the comparison of results is becoming increasingly difficult,since there are different recommendation frameworks and different settings inthe design and implementation of the experiments. Furthermore, there might beminor differences on algorithm implementation among the differentframeworks. In this paper, we compare well known recommendationalgorithms, using the same dataset, metrics and overall settings, the results ofwhich point to result differences across frameworks with the exact samesettings. Hence, we propose the use of standards that should be followed asguidelines to ensure the replication of experiments and the reproducibility ofthe results.
Original languageEnglish
Title of host publication14th International Conference on Artificial Intelligence Applications and Innovations
Place of PublicationGermany
PublisherSpringer-Verlag
Pages401-409
Number of pages9
Volume519
Publication statusPublished - 22 May 2018
Event14th International Conference on Artificial Intelligence Applications and Innovations - Rhodes, Greece, 25-27 May 2018
Duration: 22 May 2018 → …

Publication series

NameIFIP Advances in Information and Communication Technology

Conference

Conference14th International Conference on Artificial Intelligence Applications and Innovations
Period22/05/18 → …

Bibliographical note

This is a post-peer-review, pre-copyedit version of an article published in IFIP Advances in Information and Communication Technology. The final authenticated version is available online at: http://dx.doi.org/10.1007/978-3-319-92007-8_34

Keywords

  • Recommender systems
  • Evaluation
  • Reproducibility
  • Replication

Fingerprint Dive into the research topics of 'Reproducibility of experiments in recommender systems evaluation'. Together they form a unique fingerprint.

  • Cite this

    Polatidis, N., Kapetanakis, S., Pimenidis, E., & Kosmidis, K. (2018). Reproducibility of experiments in recommender systems evaluation. In 14th International Conference on Artificial Intelligence Applications and Innovations (Vol. 519, pp. 401-409). (IFIP Advances in Information and Communication Technology). Springer-Verlag. https://link.springer.com/chapter/10.1007/978-3-319-92007-8_34