Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION

We implement a variance-based distance metric (D-n) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The D-n metric is a gamma-distributed statistic that is more general than the (2) statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The D-n statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the D-n metric as a cost function and incorporating model parametric uncertainty as part of a scheme to optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness. Plain Language Summary We proposed a statistically robust distance metric to objectively quantify the skill of sea ice models in reproducing observations. The scheme is more general than other statistics (Chi2) in that it can incorporate both model and observational uncertainties in the analysis and does not require assumptions that are hard to hold for computer models, for instance, that the models are unbiased and mismatches with observations only arise from observational errors. We applied this scheme to evaluate the Los Alamos Sea Ice model with respect to its uncertainty in 40 parameters, and were able to rank 398 different model configurations according to their skill in simultaneously simulate the arge scale distributions of sea ice concentration and thickness during 2003-2009.