Residual feed intake (RFI) is a measure of feed efficiency, in which low RFI denotes improved feed efficiency. Caloric restriction (CR) is associated with feed efficiency in livestock species and to human health benefits, such as longevity and cancer prevention. We have developed pig lines that differ in RFI, and we are interested in identifying the genes and pathways that underlie feed efficiency. Prepubertal Yorkshire gilts with low RFI (n = 10) or high RFI (n = 10) were fed ad libitum or fed at restricted intake of 80% of maintenance energy requirements for 8 days. We measured serum metabolites and hormones and generated transcriptional profiles of liver and subcutaneous adipose tissue on these animals. Overall, 6,114 genes in fat and 305 genes in liver were differentially expressed (DE) in response to CR, and 311 genes in fat and 147 genes in liver were DE due to RFI differences. Pathway analyses of CR-induced DE genes indicated a dramatic switch to a conservation mode of energy usage by down-regulating lipogenesis and steroidogenesis in both liver and fat. Interestingly, CR altered expression of genes in immune and cell cycle/apoptotic pathways in fat, which may explain part of the CR-driven lifespan enhancement. In silico analysis of transcription factors revealed ESR1 as a putative regulator of the adaptive response to CR, as several targets of ESR1 in our DE fat genes were annotated as cell cycle/apoptosis genes. The lipid metabolic pathway was overrepresented by down-regulated genes due to both CR and low RFI. We propose a common energy conservation mechanism, which may be controlled by PPARA, PPARG, and/or CREB in both CR and feed-efficient pigs.