This multi-site, quasi-experimental study examined the performance outcomes of nurses (n = 152) in a military nurse transition program. A modified-performance instrument was used to assess participants in two high-fidelity simulation scenarios. Although results indicated a significant increase in scores posttraining, only moderate interrater reliability results were found for the new instrument. These findings have implications for nurse educators assessing performance-based outcomes of new nurses completing transition programs.