Specialist experts for prediction with side information

Yuri Kalnishkan, Dmitry Adamskiy, Alexey Chernov, Tim Scarfe

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBNResearchpeer-review

Abstract

The paper proposes the vicinities merging algorithm for prediction with side information. The algorithm is based on specialist experts techniques. We use vicinities in the side information domain to identify relevant past examples, apply standard learning techniques to them, and then use prediction with expert advice tools to merge those predictions. Guarantees from the theory of prediction with expert advice ensure that helpful vicinities are selected dynamically. The algorithm automatically converges on the right vicinities from an initial broad selection. We apply the resulting algorithms to two problems, prediction of implied volatility of options and prediction of students' performance at tests. On the problem of predicting implied volatility, the algorithm consistently outperforms naive competitors and a highly-tuned proprietary method used in the industry. When applied to the students' performance, the algorithm never falls behind the baseline and outperforms it when the side information is beneficial.
Original languageEnglish
Title of host publicationProccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops
Place of PublicationIEEE
Pages1470-1477
Number of pages8
Publication statusPublished - 4 Feb 2016
EventProceedings of the 2015 IEEE 15th International Conference on Data Mining Workshops - Atlantic City, NJ, USA, 14-17 November 2015
Duration: 4 Feb 2016 → …

Publication series

NameIEEE International Conference on Data Mining (ICDM)

Workshop

WorkshopProceedings of the 2015 IEEE 15th International Conference on Data Mining Workshops
Period4/02/16 → …

Fingerprint

Students
Merging
Industry

Cite this

Kalnishkan, Y., Adamskiy, D., Chernov, A., & Scarfe, T. (2016). Specialist experts for prediction with side information. In Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops (pp. 1470-1477). (IEEE International Conference on Data Mining (ICDM)). IEEE.
Kalnishkan, Yuri ; Adamskiy, Dmitry ; Chernov, Alexey ; Scarfe, Tim. / Specialist experts for prediction with side information. Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops. IEEE, 2016. pp. 1470-1477 (IEEE International Conference on Data Mining (ICDM)).
@inproceedings{72e648aa5634486f9be516893b098b96,
title = "Specialist experts for prediction with side information",
abstract = "The paper proposes the vicinities merging algorithm for prediction with side information. The algorithm is based on specialist experts techniques. We use vicinities in the side information domain to identify relevant past examples, apply standard learning techniques to them, and then use prediction with expert advice tools to merge those predictions. Guarantees from the theory of prediction with expert advice ensure that helpful vicinities are selected dynamically. The algorithm automatically converges on the right vicinities from an initial broad selection. We apply the resulting algorithms to two problems, prediction of implied volatility of options and prediction of students' performance at tests. On the problem of predicting implied volatility, the algorithm consistently outperforms naive competitors and a highly-tuned proprietary method used in the industry. When applied to the students' performance, the algorithm never falls behind the baseline and outperforms it when the side information is beneficial.",
author = "Yuri Kalnishkan and Dmitry Adamskiy and Alexey Chernov and Tim Scarfe",
year = "2016",
month = "2",
day = "4",
language = "English",
isbn = "9781467384926",
series = "IEEE International Conference on Data Mining (ICDM)",
pages = "1470--1477",
booktitle = "Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops",

}

Kalnishkan, Y, Adamskiy, D, Chernov, A & Scarfe, T 2016, Specialist experts for prediction with side information. in Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops. IEEE International Conference on Data Mining (ICDM), IEEE, pp. 1470-1477, Proceedings of the 2015 IEEE 15th International Conference on Data Mining Workshops, 4/02/16.

Specialist experts for prediction with side information. / Kalnishkan, Yuri; Adamskiy, Dmitry; Chernov, Alexey; Scarfe, Tim.

Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops. IEEE, 2016. p. 1470-1477 (IEEE International Conference on Data Mining (ICDM)).

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBNResearchpeer-review

TY - GEN

T1 - Specialist experts for prediction with side information

AU - Kalnishkan, Yuri

AU - Adamskiy, Dmitry

AU - Chernov, Alexey

AU - Scarfe, Tim

PY - 2016/2/4

Y1 - 2016/2/4

N2 - The paper proposes the vicinities merging algorithm for prediction with side information. The algorithm is based on specialist experts techniques. We use vicinities in the side information domain to identify relevant past examples, apply standard learning techniques to them, and then use prediction with expert advice tools to merge those predictions. Guarantees from the theory of prediction with expert advice ensure that helpful vicinities are selected dynamically. The algorithm automatically converges on the right vicinities from an initial broad selection. We apply the resulting algorithms to two problems, prediction of implied volatility of options and prediction of students' performance at tests. On the problem of predicting implied volatility, the algorithm consistently outperforms naive competitors and a highly-tuned proprietary method used in the industry. When applied to the students' performance, the algorithm never falls behind the baseline and outperforms it when the side information is beneficial.

AB - The paper proposes the vicinities merging algorithm for prediction with side information. The algorithm is based on specialist experts techniques. We use vicinities in the side information domain to identify relevant past examples, apply standard learning techniques to them, and then use prediction with expert advice tools to merge those predictions. Guarantees from the theory of prediction with expert advice ensure that helpful vicinities are selected dynamically. The algorithm automatically converges on the right vicinities from an initial broad selection. We apply the resulting algorithms to two problems, prediction of implied volatility of options and prediction of students' performance at tests. On the problem of predicting implied volatility, the algorithm consistently outperforms naive competitors and a highly-tuned proprietary method used in the industry. When applied to the students' performance, the algorithm never falls behind the baseline and outperforms it when the side information is beneficial.

M3 - Conference contribution with ISSN or ISBN

SN - 9781467384926

T3 - IEEE International Conference on Data Mining (ICDM)

SP - 1470

EP - 1477

BT - Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops

CY - IEEE

ER -

Kalnishkan Y, Adamskiy D, Chernov A, Scarfe T. Specialist experts for prediction with side information. In Proccedings of the 2015 IEEE 15th International Conference on Data Mining Workshops. IEEE. 2016. p. 1470-1477. (IEEE International Conference on Data Mining (ICDM)).