Residual feed intake (RFI) is the difference between observed and predicted feed intake of an animal, based on growth and maintenance requirements. In Yorkshire pigs, divergent selection for increased (Low RFI) and decreased (High RFI) RFI was carried out over 10 generations (G) while feeding a corn- and soybean-meal-based, higher-energy, lower-fiber (HELF) diet. In G8 to G10, representing 4 replicates, barrows and gilts (n = 649) of the RFI lines were fed the HELF diet and a diet incorporating coproducts that were lower in energy and higher in dietary fiber (LEHF). The diets differed in ME, 3.32 vs. 2.87 Mcal/kg, and in neutral detergent fiber (NDF), 9.4% vs. 25.9%, respectively. The impact of the LEHF diet on 1) performance and growth, 2) diet digestibility, 3) genetic parameter estimates, and 4) responses to selection for RFI, when fed the HELF, was assessed. In general, the LEHF diet reduced the performance of both lines. When fed the HELF diet, the Low RFI pigs had lower (P < 0.05) ADFI (−12%), energy intake (−12%), ADG (−6%), and backfat depth (−12%); similar (P > 0.05) loin muscle area (LMA; +5%); and greater (P < 0.05) feed efficiency (i.e., 8% higher G:F and 7% lower RFI) than the High RFI line. These patterns of line differences were still present under the LEHF diet but differences for ADFI (−11%), energy intake (−10%), G:F (+2%), and RFI (−6%) were reduced compared to the HELF diet. Apparent total tract digestibility (ATTD) of the HELF and LEHF diets was assessed using 116 barrows and gilts from G8. When fed the HELF diet, ATTD of DM, GE, N, and NDF were similar between lines (P ≥ 0.27), but when fed the LEHF diet, the Low RFI pigs had greater digestibility (7%, 7%, 10%, and 32%) than the High RFI line (P ≤ 0.04). To measure responses to selection for RFI and estimate genetic parameters, data from all 10 generations were used (HELF; n = 2,310; LEHF, n = 317). Heritability estimates of performance traits ranged from 0.19 to 0.63, and genetic correlations of traits between diets were high and positive, ranging from 0.87 (RFI) to 0.99 (LMA). By G10, RFI in the Low RFI line was 3.86 and 1.50 genetic SD lower than in the High RFI line when fed the HELF and LEHF diets, respectively. Taken together, the results of this study demonstrate that responses to selection for RFI when fed a HELF diet are not fully realized when pigs are fed an extremely LEHF diet. Thus, feeding diets that differ from those used for selection may not maximize genetic potential for feed efficiency.