Anderson, J. L., and W. F. Stern, 1996: Evaluating the potential predictive utility of ensemble forecasts. Journal of Climate, 9(2), 260-269.

Abstract: A method is presented for determining when an ensemble of model forecasts has the potential to provide some useful information. An ensemble forecast of a particular scalar quantity is said to have potential predictive utility when the ensemble forecast distribution is significantly different from an appropriate climatological distribution. Here, the potential predictive utility is measured using Kuiper's statistical test for comparing two discrete distributions. More traditional measures of the potential usefulness of an ensemble forecast based on ensemble mean or variance discard possibly valuable information by making implicit assumptions about the distributions being compared.

Application of the potential predictive utility to long integrations of an atmospheric general circulation model in a boundary value problem (an ensemble of Atmospheric Model Intercomparison Project integrations) reveals a number of features about the response of a GCM to observed sea surface temperatures. In particular, the ensemble of forecasts is found to have potential predictive utility over large geographic areas for a number of atmospheric fields during strong El Niño-Southern Oscillation anomalous events. Unfortunately, there are only limited areas of potential predictive utility for near-surface fields and precipitation outside the regions of the tropical oceans. Nevertheless, the method presented here can identify all areas where the GCM ensemble may provide useful information, whereas methods that make assumptions about the distribution of the ensemble forecast variables may not be able to do so.