Prediction with Expert Evaluators' Advice

Alexey Chernov, Vladimir Vovk

    Research output: Chapter in Book/Report/Conference proceedingConference contribution with ISSN or ISBN

    Abstract

    We introduce a new protocol for prediction with expert advice in which each expert evaluates the learner’s and his own performance using a loss function that may change over time and may be different from the loss functions used by the other experts. The learner’s goal is to perform better or not much worse than each expert, as evaluated by that expert, for all experts simultaneously. If the loss functions used by the experts are all proper scoring rules and all mixable, we show that the defensive forecasting algorithm enjoys the same performance guaranteeas that attainable by the Aggregating Algorithm in the standard setting and known to be optimal. This result is also applied to the case of “specialist” experts. In this case, the defensive forecasting algorithm reduces to a simple modification of the Aggregating Algorithm.
    LanguageEnglish
    Title of host publication20th International Conference, ALT 2009
    Place of PublicationBerlin
    Pages8-22
    Number of pages15
    Volume5809
    DOIs
    StatePublished - 31 Dec 2009
    Event20th International Conference, ALT 2009 - Porto, Portugal, October 3-5, 2009
    Duration: 31 Dec 2009 → …

    Publication series

    NameLecture Notes in Computer Science

    Conference

    Conference20th International Conference, ALT 2009
    Period31/12/09 → …

    Fingerprint

    Evaluator
    Prediction
    Loss function
    Scoring rules

    Bibliographical note

    © Springer-Verlag Berlin Heidelberg 2009

    Cite this

    Chernov, A., & Vovk, V. (2009). Prediction with Expert Evaluators' Advice. In 20th International Conference, ALT 2009 (Vol. 5809, pp. 8-22). (Lecture Notes in Computer Science). Berlin. DOI: 10.1007/978-3-642-04414-4_6
    Chernov, Alexey ; Vovk, Vladimir. / Prediction with Expert Evaluators' Advice. 20th International Conference, ALT 2009. Vol. 5809 Berlin, 2009. pp. 8-22 (Lecture Notes in Computer Science).
    @inproceedings{407d6d832c3f4f4fb9b86f7e1d4c99f7,
    title = "Prediction with Expert Evaluators' Advice",
    abstract = "We introduce a new protocol for prediction with expert advice in which each expert evaluates the learner’s and his own performance using a loss function that may change over time and may be different from the loss functions used by the other experts. The learner’s goal is to perform better or not much worse than each expert, as evaluated by that expert, for all experts simultaneously. If the loss functions used by the experts are all proper scoring rules and all mixable, we show that the defensive forecasting algorithm enjoys the same performance guaranteeas that attainable by the Aggregating Algorithm in the standard setting and known to be optimal. This result is also applied to the case of “specialist” experts. In this case, the defensive forecasting algorithm reduces to a simple modification of the Aggregating Algorithm.",
    author = "Alexey Chernov and Vladimir Vovk",
    note = "\{circledC} Springer-Verlag Berlin Heidelberg 2009",
    year = "2009",
    month = "12",
    day = "31",
    doi = "10.1007/978-3-642-04414-4_6",
    language = "English",
    volume = "5809",
    series = "Lecture Notes in Computer Science",
    pages = "8--22",
    booktitle = "20th International Conference, ALT 2009",

    }

    Chernov, A & Vovk, V 2009, Prediction with Expert Evaluators' Advice. in 20th International Conference, ALT 2009. vol. 5809, Lecture Notes in Computer Science, Berlin, pp. 8-22, 20th International Conference, ALT 2009, 31/12/09. DOI: 10.1007/978-3-642-04414-4_6

    Prediction with Expert Evaluators' Advice. / Chernov, Alexey; Vovk, Vladimir.

    20th International Conference, ALT 2009. Vol. 5809 Berlin, 2009. p. 8-22 (Lecture Notes in Computer Science).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution with ISSN or ISBN

    TY - GEN

    T1 - Prediction with Expert Evaluators' Advice

    AU - Chernov,Alexey

    AU - Vovk,Vladimir

    N1 - © Springer-Verlag Berlin Heidelberg 2009

    PY - 2009/12/31

    Y1 - 2009/12/31

    N2 - We introduce a new protocol for prediction with expert advice in which each expert evaluates the learner’s and his own performance using a loss function that may change over time and may be different from the loss functions used by the other experts. The learner’s goal is to perform better or not much worse than each expert, as evaluated by that expert, for all experts simultaneously. If the loss functions used by the experts are all proper scoring rules and all mixable, we show that the defensive forecasting algorithm enjoys the same performance guaranteeas that attainable by the Aggregating Algorithm in the standard setting and known to be optimal. This result is also applied to the case of “specialist” experts. In this case, the defensive forecasting algorithm reduces to a simple modification of the Aggregating Algorithm.

    AB - We introduce a new protocol for prediction with expert advice in which each expert evaluates the learner’s and his own performance using a loss function that may change over time and may be different from the loss functions used by the other experts. The learner’s goal is to perform better or not much worse than each expert, as evaluated by that expert, for all experts simultaneously. If the loss functions used by the experts are all proper scoring rules and all mixable, we show that the defensive forecasting algorithm enjoys the same performance guaranteeas that attainable by the Aggregating Algorithm in the standard setting and known to be optimal. This result is also applied to the case of “specialist” experts. In this case, the defensive forecasting algorithm reduces to a simple modification of the Aggregating Algorithm.

    U2 - 10.1007/978-3-642-04414-4_6

    DO - 10.1007/978-3-642-04414-4_6

    M3 - Conference contribution with ISSN or ISBN

    VL - 5809

    T3 - Lecture Notes in Computer Science

    SP - 8

    EP - 22

    BT - 20th International Conference, ALT 2009

    CY - Berlin

    ER -

    Chernov A, Vovk V. Prediction with Expert Evaluators' Advice. In 20th International Conference, ALT 2009. Vol. 5809. Berlin. 2009. p. 8-22. (Lecture Notes in Computer Science). Available from, DOI: 10.1007/978-3-642-04414-4_6