Skip to main content
Neighborhood-Based Verification of Precipitation Forecasts from Convection-Allowing NCAR WRF Model Simulations and the Operational NAM
Weather and Forecasting
  • Adam J. Clark, National Oceanic and Atmospheric Administration
  • William A. Gallus, Jr., Iowa State University
  • Morris L. Weisman, National Center for Atmospheric Research
Document Type
Publication Version
Published Version
Publication Date

Since 2003 the National Center for Atmospheric Research (NCAR) has been running various experimental convection-allowing configurations of the Weather Research and Forecasting Model (WRF) for domains covering a large portion of the central United States during the warm season (April–July). In this study, the skill of 3-hourly accumulated precipitation forecasts from a large sample of these convection-allowing simulations conducted during 2004–05 and 2007–08 is compared to that from operational North American Mesoscale (NAM) model forecasts using a neighborhood-based equitable threat score (ETS). Separate analyses were conducted for simulations run before and after the implementation in 2007 of positive-definite (PD) moisture transport for the NCAR-WRF simulations. The neighborhood-based ETS (denoted hETSir) relaxes the criteria for ‘‘hits’’ (i.e., correct forecasts) by considering grid points within a specified radius r. It is shown that hETSir is more useful than the traditional ETS because hETSir can be used to diagnose differences in precipitation forecast skill between different models as a function of spatial scale, whereas the traditional ETS only considers the spatial scale of the verification grid. It was found that differences in hETSir between NCAR-WRF and NAM generally increased with increasing r, with NCAR-WRF having higher scores. Examining time series of hETSir for r 5 100 and r 5 0 km (which simply reduces to the ‘‘traditional’’ ETS), statistically significant differences between NCAR-WRF and NAM were found at many forecast lead times for hETSi100 but only a few times for hETSi0. Larger and more statistically significant differences occurred with the 2007–08 cases relative to the 2004–05 cases. Because of differences in model configurations and dominant large-scale weather regimes, a more controlled experiment would have been needed to diagnose the reason for the larger differences that occurred with the 2007–08 cases. Finally, a compositing technique was used to diagnose the differences in the spatial distribution of the forecasts. This technique implied westward displacement errors for NAM model forecasts in both sets of cases and in NCAR-WRF model forecasts for the 2007–08 cases. Generally, the results are encouraging because they imply that advantages in convectionallowing relative to convection-parameterizing simulations noted in recent studies are reflected in an objective neighborhood-based metric.


This article is from Weather and Forecasting 25 (2010): 1495, doi: 10.1175/2010WAF2222404.1. Posted with permission.

Copyright Owner
American Meteorological Society
File Format
Citation Information
Adam J. Clark, William A. Gallus and Morris L. Weisman. "Neighborhood-Based Verification of Precipitation Forecasts from Convection-Allowing NCAR WRF Model Simulations and the Operational NAM" Weather and Forecasting Vol. 25 Iss. 5 (2010) p. 1495 - 1509
Available at: