A high spatial resolution model of energy use in residential buildings was developed based on time use data. The development of this model was guided by a theoretical framework that explains the nature of a dwelling’s physical characteristics, the nature of its occupants’ energy use behaviors and the ways in which the dwelling and its occupants interact to determine the energy use of a household. Energy Plus was used as the model platform. In the model, the occupants’ domestic activity pattern was extracted from American Time Use Survey (ATUS) data via bootstrap sampling. The dwelling’s physical characteristics were based on real world scenarios. Virtual experiments with 3- to 5-occupant household compositions were conducted to examine the model properties. Simulation results show that (1) bell-shaped distributions were present in annual heating load demands for all household compositions, (2) the load demands for different batches of samples of the same household composition demonstrated a narrow range of variations, and (3) the simulated hourly appliance and lighting load profiles were in agreement with those generated from field-metered data at both whole-house and sub-house levels. These results indicate the model’s overall robustness and verify its ability to simulate realistic residential energy use load profiles.