The increasing penetration of variable and intermittent generation, especially the wind power, are introducing greater uncertainties to the modern power systems. The operation of power system is thus facing more challenges. Many effective models and algorithms are proposed to mitigate the challenges. Among those, the Stochastic Security-Constrained Unit Commitment (S-SCUC) is one of the most important tools. The S-SCUC has many different features over the traditional deterministic Security-Constrained Unit Commitment (D-SCUC). The propose of this paper is to develop a systematic testing framework to compare the performances of S-SCUC and D-SCUC. To make the results better reflecting the true power systems, we use the real generation data from the ISO New England (ISO-NE) to conduct the tests. The ISO-NE bulk network is condensed into a 8-zone economic test system. Computational experiments are designed to facilitate the comparisons under a variety of possible system conditions, including various wind power penetration levels, various reserve requirement levels, and various months of a year. The testing results have demonstrated that the average cost saving of S-SCUC is a "U-shape" function, where S-SCUC has the least cost saving over D-SCUC at the saddle point of the "U-shape".