Using a full three-dimensional (3D), quantum transport simulator, we theoretically investigate the effects of surface roughness scattering (SRS) on the device characteristics of Si nanowire transistors (SNWTs). The microscopic structure of the Si/SiO2 interface roughness is directly treated by using a 3D finite element technique. The results show that (1) SRS reduces the electron density of states in the channel, which increases the SNWT threshold voltage, and (2) the SRS in SNWTs becomes less effective when fewer propagating modes are occupied, which implies that SRS is less important in small-diameter SNWTs with few modes conducting than in planar metal-oxide-semiconductor field-effect-transistors with many transverse modes occupied.