Heat stress (HS) is one of the costliest issues in the U.S. pork industry. Aims of the present study were to determine the consequences of repeated exposure to HS on growth performance, and the effects of a high fiber diet, the genetic potential for high lean tissue accretion, and the genetic potential for residual feed intake (RFI) on resilience to HS. Barrows (n = 97) from three genetic lines (commercial, high RFI, low RFI) where subjected three times to a 4-day HS treatment (HS1, HS2, and HS3) which was preceded by a 9-day neutral (TN) adaptation period (TN1) and alternated by 7-day periods of neutral temperatures (TN2, TN3, and TN4). Body weight gain (BWG), feed intake (FI), feed conversion efficiency (FCE), RFI, and the drop in BWG and FI between TN and HS were estimated for each period, and slaughter traits were measured at the end of TN4. Commercial pigs had lower FI when fed a high fiber diet compared to a regular diet (2.70 ± 0.08 vs. 2.96 ± 0.08 kg/d; P < 0.05), while no differences were found for BWG, RFI or FCE. HS reduced FI, BWG, and FCE, increased RFI, and resulted in leaner pigs that generate smaller carcasses at slaughter. In TN, commercial pigs grew faster than the low and high RFI pigs (1.22 ± 0.06 vs. 0.720 ± 0.05 and 0.657 ± 0.07; P < 0.001) but growth rates were not significantly different between the lines during HS. Growth rates for the low RFI and high RFI pigs were similar both during TN and during HS. Pigs of interest for genetic improvement are those that are able to maintain growth rates during HS. Our results show that response in growth to HS was repeatable over subsequent 4-d HS cycles, which suggests the potential for including this response in the breeding index. The best performing animals during HS are likely those that are not highly superior for growth in TN.