Probabilistic wind power scenarios constitute a crucial input for stochastic day-ahead unit commitment in power systems with deep penetration of wind generation. To minimize the expected cost, the scenario time series of wind power amounts available should accurately represent the stochastic process for available wind power as it is estimated on the day ahead. The high computational demands of stochastic programming motivate a search for ways to evaluate scenarios without extensively simulating the stochastic unit commitment procedure. Reliability of wind power scenario sets can be assessed by statistical verification approaches. In this study, we examine the relationship between the statistical evaluation metrics and the results of stochastic unit commitment. Lack of uniformity in a mass transportation distance rank histogram can eliminate scenario sets that might lead to either excessive no-load costs of committed units or high penalty costs for violating energy balance. Event-based metrics can help to predict the cost performance of the remaining scenario sets.