Towards Reliable Benchmarking of Solar Flare Forecasting Methods

Abstract

Solar flares occur in complex sunspot groups, but it remains unclear how the probability of pro- ducing a flare of a given magnitude relates to the characteristics of the sunspot group. Here, we use Geostationary Operational Environment Satellite X-ray flares and McIntosh group classifications from solar cycles 21 and 22 to calculate average flare rates for each McIntosh class and use these to deter- mine Poisson probabilities for different flare magnitudes. Forecast verification measures are studied to find optimum thresholds to convert Poisson flare probabilities into yes/no predictions of cycle 23 flares. A case is presented to adopt the true skill statistic (TSS) as a standard for forecast comparison over the commonly used Heidke skill score (HSS). In predicting flares over 24 hr, the maximum values of TSS achieved are 0.44 (C-class), 0.53 (M-class), 0.74 (X-class), 0.54 (?M1.0), and 0.46 (?C1.0). The maximum values of HSS are 0.38 (C-class), 0.27 (M-class), 0.14 (X-class), 0.28 (?M1.0), and 0.41 (?C1.0). These show that Poisson probabilities perform comparably to some more complex predic- tion systems, but the overall inaccuracy highlights the problem with using average values to represent flaring rate distributions.oups