Calibrating Probability Density Forecasts with Multi-objective Search
File Type:
PDFItem Type:
Technical ReportDate:
2006-02-10Citation:
Carney, Michael; Cunningham, Padraig. 'Calibrating Probability Density Forecasts with Multi-objective Search'. - Dublin, Trinity College Dublin, Department of Computer Science, TCD-CS-2006-07, 2006, pp12Download Item:
TCD-CS-2006-07.pdf (PDF) 290.4Kb
Abstract:
In this paper, we show that the optimization of density forecasting models for regression in machine learning can be formulated as a
multi-objective problem.We describe the two objectives of sharpness and
calibration and suggest suitable scoring
metrics for both.We use the popular negative log-likelihood as a measure of sharpness and the probability
integral transform as a measure of calibration. We show how optimization on negative log-likelihood alone often results in sub-optimal models.
To solve this problem we introduce a multi-objective evolutionary optimization framework that can produce better density forecasts from a
prediction users perspective. Our experiments show improvements over
state-of-the-art approaches.
Author: Carney, Michael; Cunningham, Padraig
Publisher:
Trinity College Dublin, Department of Computer ScienceType of material:
Technical ReportCollections:
Series/Report no:
Computer Science Technical ReportTCD-CS-2006-07
Availability:
Full text availableKeywords:
Computer ScienceLicences: