We present new calibrations of the near-infrared (near-IR) surface brightness fluctuation (SBF) distance method for the F110W (J110) and F160W (H160) bandpasses of the Wide Field Camera 3 Infrared Channel (WFC3/IR) on the Hubble Space Telescope. The calibrations are based on data for 16 early-type galaxies in the Virgo and Fornax clusters observed with WFC3/IR and are provided as functions of both the optical (g z ) 475- 850 and near-infrared (J H 110 160 - ) colors. The scatter about the linear calibration relations for the luminous red galaxies in the sample is approximately 0.10 mag, corresponding to a statistical error of 5% in distance. Our results imply that the distance to any suitably bright elliptical galaxy can be measured with this precision out to about 80 Mpc in a single-orbit observation with WFC3/IR, making this a remarkably powerful instrument for extragalactic distances. The calibration sample also includes much bluer and lower-luminosity galaxies than previously used for IR SBF studies, revealing interesting population differences that cause the calibration scatter to increase for dwarf galaxies. Comparisons with single-burst population models show that as expected, the redder early-type galaxies contain old, metal-rich populations, while the bluer dwarf ellipticals contain a wider range of ages and lower metallicities than their more massive counterparts. Radial SBF gradients reveal that IR color gradients are largely an age effect; the bluer dwarfs typically have their youngest populations near their centers, while the redder giant ellipticals show only weak trends and in the opposite sense. Because of the population variations among bluer galaxies, distance measurements in the near-IR are best limited to red early-type galaxies. We conclude with some practical guidelines for using WFC3/IR to measure reliable SBF distances.