Probabilistic wind power scenarios constitute a crucial input for stochastic day-ahead unit commitment in power systems with deep penetration of wind generation. To minimize the cost of implemented solutions, the scenario time series of wind power amounts available should accurately represent the stochastic process for available wind power as it is estimated on the day ahead. The high computational demands of stochastic programming motivate a search for ways to evaluate scenarios without extensively simulating the stochastic unit commitment procedure. The statistical reliability of wind power scenario sets can be assessed by approaches extended from ensemble forecast verification. We examine the relationship between the statistical reliability metrics and the results of stochastic unit commitment when implemented solutions encounter the observed available wind power. Lack of uniformity in a mass transportation distance rank histogram can eliminate scenario sets that might lead to either excessive no-load costs of committed units or high penalty costs for violating energy balance when the committed units are dispatched. Event-based metrics can help to predict results of implementing solutions found with the remaining scenario sets.