The selection, introduction, and cultivation of non-native woody plants beyond their native ranges can have great benefits, but also unintended consequences. Among these consequences is the tendency for some species to naturalize and become invasive pests in new environments to which they were introduced. In lieu of lengthy and costly field trials, risk-assessment models can be used to predict the likelihood of naturalization. We compared the relative performance of five established risk-assessment models on species datasets from two previously untested areas: southern Minnesota and northern Missouri. Model classification rates ranged from 64.2 to 90.5%, biologically significant errors ranged from 4.4 to 9.3%, and horticulturally limiting errors ranged from 6.6 to 30.4%. For the random forest model, we investigated the importance of variables used to predict naturalization by examining datasets for five distinct study areas across the Upper Midwest. Geographic-risk ratios were the most important predictors of species' tendency to naturalize. Other factors, such as quick maturity, record of invading elsewhere, and production of fleshy, bird-dispersed fruit were also important in the random forest models. Although some models tested need additional refinement, the random forest models maintain robustness and provide additional information on plant-specific characteristics that contribute to naturalization.